[v3] getrandom system call wrapper [BZ #17252]
Commit Message
Here is a new attempt at providing the getrandom system call.
I have retained the __getrandom symbol mangling. The justification for
that is that getrandom is a fairly common name. Application code might
use it for something else entirely and interpose their definition, so
that libraries cannot rely on it doing the right thing. I think the
mangling is justified because it is hard to spot that getrandom is
broken due to interposition. As <sys/random.h> is a new header, the
macro will be exposed to few applications.
I have made the system call wrapper a cancellation point. (If we
implement the simpler getentropy interface, it would not be a
cancellation point.)
Thanks,
Florian
Comments
On Sep 08 2016, Florian Weimer <fweimer@redhat.com> wrote:
> I have retained the __getrandom symbol mangling. The justification for
> that is that getrandom is a fairly common name. Application code might
> use it for something else entirely and interpose their definition, so that
> libraries cannot rely on it doing the right thing. I think the mangling
> is justified because it is hard to spot that getrandom is broken due to
> interposition. As <sys/random.h> is a new header, the macro will be
> exposed to few applications.
It makes it impossible to take the address of the function, though.
Andreas.
On 09/08/2016 03:01 PM, Andreas Schwab wrote:
> On Sep 08 2016, Florian Weimer <fweimer@redhat.com> wrote:
>
>> I have retained the __getrandom symbol mangling. The justification for
>> that is that getrandom is a fairly common name. Application code might
>> use it for something else entirely and interpose their definition, so that
>> libraries cannot rely on it doing the right thing. I think the mangling
>> is justified because it is hard to spot that getrandom is broken due to
>> interposition. As <sys/random.h> is a new header, the macro will be
>> exposed to few applications.
>
> It makes it impossible to take the address of the function, though.
It does. Do you have a better suggestion to achieve the same effect?
Thanks,
Florian
On Sep 08 2016, Florian Weimer <fweimer@redhat.com> wrote:
> On 09/08/2016 03:01 PM, Andreas Schwab wrote:
>> On Sep 08 2016, Florian Weimer <fweimer@redhat.com> wrote:
>>
>>> I have retained the __getrandom symbol mangling. The justification for
>>> that is that getrandom is a fairly common name. Application code might
>>> use it for something else entirely and interpose their definition, so that
>>> libraries cannot rely on it doing the right thing. I think the mangling
>>> is justified because it is hard to spot that getrandom is broken due to
>>> interposition. As <sys/random.h> is a new header, the macro will be
>>> exposed to few applications.
>>
>> It makes it impossible to take the address of the function, though.
>
> It does. Do you have a better suggestion to achieve the same effect?
If getrandom is ever standardized be POSIX this will have to be reverted
anyway.
Andreas.
On 09/08/2016 03:19 PM, Andreas Schwab wrote:
> On Sep 08 2016, Florian Weimer <fweimer@redhat.com> wrote:
>
>> On 09/08/2016 03:01 PM, Andreas Schwab wrote:
>>> On Sep 08 2016, Florian Weimer <fweimer@redhat.com> wrote:
>>>
>>>> I have retained the __getrandom symbol mangling. The justification for
>>>> that is that getrandom is a fairly common name. Application code might
>>>> use it for something else entirely and interpose their definition, so that
>>>> libraries cannot rely on it doing the right thing. I think the mangling
>>>> is justified because it is hard to spot that getrandom is broken due to
>>>> interposition. As <sys/random.h> is a new header, the macro will be
>>>> exposed to few applications.
>>>
>>> It makes it impossible to take the address of the function, though.
>>
>> It does. Do you have a better suggestion to achieve the same effect?
>
> If getrandom is ever standardized be POSIX this will have to be reverted
> anyway.
So you don't object to this approach?
We could keep it even in case of standardization under _GNU_SOURCE, or
with _FORTIFY_SOURCE. POSIX also seems to allow a function-style macro
in addition to the function, so we could add a prototype for getrandom
and an alias.
Thanks,
Florian
On Sep 08 2016, Florian Weimer <fweimer@redhat.com> wrote:
> On 09/08/2016 03:19 PM, Andreas Schwab wrote:
>> On Sep 08 2016, Florian Weimer <fweimer@redhat.com> wrote:
>>
>>> On 09/08/2016 03:01 PM, Andreas Schwab wrote:
>>>> On Sep 08 2016, Florian Weimer <fweimer@redhat.com> wrote:
>>>>
>>>>> I have retained the __getrandom symbol mangling. The justification for
>>>>> that is that getrandom is a fairly common name. Application code might
>>>>> use it for something else entirely and interpose their definition, so that
>>>>> libraries cannot rely on it doing the right thing. I think the mangling
>>>>> is justified because it is hard to spot that getrandom is broken due to
>>>>> interposition. As <sys/random.h> is a new header, the macro will be
>>>>> exposed to few applications.
>>>>
>>>> It makes it impossible to take the address of the function, though.
>>>
>>> It does. Do you have a better suggestion to achieve the same effect?
>>
>> If getrandom is ever standardized be POSIX this will have to be reverted
>> anyway.
>
> So you don't object to this approach?
Not providing a getrandom symbol is a bug.
Andreas.
On Thu, Sep 8, 2016 at 9:19 AM, Andreas Schwab <schwab@suse.de> wrote:
>
> If getrandom is ever standardized be POSIX this will have to be reverted
> anyway.
If we're worried about that, maybe the thin syscall wrapper should be
called linux_getrandom? I think it will be useful to have a thin
syscall wrapper for this, independent of whether we also supply some
hypothetical POSIX-conformant getrandom(), or BSD-compatible
arc4random(), or whatever.
Reserving linux_* for thin wrappers around Linux-specific system calls
also might break the consensus deadlock around the
future-compatibility implications of other such wrappers.
zw
On 09/08/2016 04:19 PM, Zack Weinberg wrote:
> On Thu, Sep 8, 2016 at 9:19 AM, Andreas Schwab <schwab@suse.de> wrote:
>>
>> If getrandom is ever standardized be POSIX this will have to be reverted
>> anyway.
>
> If we're worried about that, maybe the thin syscall wrapper should be
> called linux_getrandom?
Solaris has already adopted getrandom in userpsace, and uses this symbol
(and also <sys/random.h>).
Florian
On Thu, Sep 8, 2016 at 10:31 AM, Florian Weimer <fweimer@redhat.com> wrote:
> On 09/08/2016 04:19 PM, Zack Weinberg wrote:
>>
>> On Thu, Sep 8, 2016 at 9:19 AM, Andreas Schwab <schwab@suse.de> wrote:
>>>
>>>
>>> If getrandom is ever standardized be POSIX this will have to be reverted
>>> anyway.
>>
>>
>> If we're worried about that, maybe the thin syscall wrapper should be
>> called linux_getrandom?
>
> Solaris has already adopted getrandom in userpsace, and uses this symbol
> (and also <sys/random.h>).
With compatible semantics?
zw
On 09/08/2016 04:39 PM, Zack Weinberg wrote:
> On Thu, Sep 8, 2016 at 10:31 AM, Florian Weimer <fweimer@redhat.com> wrote:
>> On 09/08/2016 04:19 PM, Zack Weinberg wrote:
>>>
>>> On Thu, Sep 8, 2016 at 9:19 AM, Andreas Schwab <schwab@suse.de> wrote:
>>>>
>>>>
>>>> If getrandom is ever standardized be POSIX this will have to be reverted
>>>> anyway.
>>>
>>>
>>> If we're worried about that, maybe the thin syscall wrapper should be
>>> called linux_getrandom?
>>
>> Solaris has already adopted getrandom in userpsace, and uses this symbol
>> (and also <sys/random.h>).
>
> With compatible semantics?
Judging by the manual page, yes.
Florian
On Thu, Sep 8, 2016 at 10:40 AM, Florian Weimer <fweimer@redhat.com> wrote:
> On 09/08/2016 04:39 PM, Zack Weinberg wrote:
>> On Thu, Sep 8, 2016 at 10:31 AM, Florian Weimer <fweimer@redhat.com>
>> wrote:
>>> On 09/08/2016 04:19 PM, Zack Weinberg wrote:
>>>>
>>>> If we're worried about [POSIX], maybe the thin syscall wrapper should be
>>>> called linux_getrandom?
>>>
>>> Solaris has already adopted getrandom in userpsace, and uses this symbol
>>> (and also <sys/random.h>).
>>
>> With compatible semantics?
>
> Judging by the manual page, yes.
OK, then I have no objection to calling the function getrandom. (I
haven't looked at the patch in detail, though.)
zw
On 09/08/2016 06:02 AM, Florian Weimer wrote:
> On 09/08/2016 03:01 PM, Andreas Schwab wrote:
>> On Sep 08 2016, Florian Weimer <fweimer@redhat.com> wrote:
>>
>>> I have retained the __getrandom symbol mangling. The justification for
>>> that is that getrandom is a fairly common name. Application code might
>>> use it for something else entirely and interpose their definition, so that
>>> libraries cannot rely on it doing the right thing. I think the mangling
>>> is justified because it is hard to spot that getrandom is broken due to
>>> interposition. As <sys/random.h> is a new header, the macro will be
>>> exposed to few applications.
>>
>> It makes it impossible to take the address of the function, though.
>
> It does. Do you have a better suggestion to achieve the same effect?
Using __asm__ to change the assembler name of the C-level symbol is probably
preferred for __GCC__. For really bare ISO C, you'd have to rely on the define.
r~
On 09/08/2016 08:28 PM, Richard Henderson wrote:
> On 09/08/2016 06:02 AM, Florian Weimer wrote:
>> On 09/08/2016 03:01 PM, Andreas Schwab wrote:
>>> On Sep 08 2016, Florian Weimer <fweimer@redhat.com> wrote:
>>>
>>>> I have retained the __getrandom symbol mangling. The justification for
>>>> that is that getrandom is a fairly common name. Application code might
>>>> use it for something else entirely and interpose their definition, so that
>>>> libraries cannot rely on it doing the right thing. I think the mangling
>>>> is justified because it is hard to spot that getrandom is broken due to
>>>> interposition. As <sys/random.h> is a new header, the macro will be
>>>> exposed to few applications.
>>>
>>> It makes it impossible to take the address of the function, though.
>>
>> It does. Do you have a better suggestion to achieve the same effect?
>
> Using __asm__ to change the assembler name of the C-level symbol is probably
> preferred for __GCC__.
This does not work in all cases because a getrandom function definition
will inherit the asm alias on the prototype and interpose the
supposedly-protected symbol.
But now that we have a <sys/random.h> header, maybe this isn't a
problem—if an application includes the <sys/random.h> header, it shows
that it is aware of getrandom, so it's less likely that it's interposing
the function accidentally.
Thanks,
Florian
On 09/08/2016 11:32 AM, Florian Weimer wrote:
>> Using __asm__ to change the assembler name of the C-level symbol is probably
>> preferred for __GCC__.
>
> This does not work in all cases because a getrandom function definition will
> inherit the asm alias on the prototype and interpose the supposedly-protected
> symbol.
>
> But now that we have a <sys/random.h> header, maybe this isn't a problem—if an
> application includes the <sys/random.h> header, it shows that it is aware of
> getrandom, so it's less likely that it's interposing the function accidentally.
Indeed, that's what I was assuming. I hadn't read the previous patches where
this header didn't exist.
r~
On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
> I have made the system call wrapper a cancellation point. (If we
> implement the simpler getentropy interface, it would not be a
> cancellation point.)
Why did you do that? Even though the system call is new, and thus can't
have been used in existing code directly, making it a cancellation point
will make all callers cancellation points too. Therefore, for example,
we couldn't use it in the implementation of any POSIX functions (that
are not cancellation points) in glibc without having to disable and
restore the cancellation state around it every time.
It might be even more convenient to have one wrapper that is a
cancellation point and one that is not.
Can't we just let cancellation rot in its corner?
On 09/09/2016 04:21 PM, Torvald Riegel wrote:
> On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
>> I have made the system call wrapper a cancellation point. (If we
>> implement the simpler getentropy interface, it would not be a
>> cancellation point.)
>
> Why did you do that?
I have to, because it can block indefinitely.
> Even though the system call is new, and thus can't
> have been used in existing code directly, making it a cancellation point
> will make all callers cancellation points too. Therefore, for example,
> we couldn't use it in the implementation of any POSIX functions (that
> are not cancellation points) in glibc without having to disable and
> restore the cancellation state around it every time.
The system call definition facility also provides a
__getrandom_notcancel entry point, which can be called if we don't
expect blocking. Obviously, this is for internal use only.
> It might be even more convenient to have one wrapper that is a
> cancellation point and one that is not.
>
> Can't we just let cancellation rot in its corner?
No, we have many customers who use it (and this despite the fact that
the current implementation has a critical race condition).
Synchronous cancellation is often easier to use than the alternative
because our API does not allow waiting for arbitrary combination of events.
Asynchronous cancellation is not very useful, though.
Thanks,
Florian
On Fri, Sep 9, 2016 at 10:28 AM, Florian Weimer <fweimer@redhat.com> wrote:
> On 09/09/2016 04:21 PM, Torvald Riegel wrote:
>> On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
>>>
>>> I have made the system call wrapper a cancellation point. (If we
>>> implement the simpler getentropy interface, it would not be a
>>> cancellation point.)
>>
>>
>> Why did you do that?
>
> I have to, because it can block indefinitely.
Is it practical to make it a cancellation point if and only if called
in blocking mode? I suspect that is the behavior basically everyone
wants. Unlike with file descriptors, blocking/nonblocking is visible
in the flags passed to the function, so it might not be hard.
(What does Solaris' implementation do? OpenBSD's?)
zw
On 09/09/2016 04:41 PM, Zack Weinberg wrote:
> On Fri, Sep 9, 2016 at 10:28 AM, Florian Weimer <fweimer@redhat.com> wrote:
>> On 09/09/2016 04:21 PM, Torvald Riegel wrote:
>>> On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
>>>>
>>>> I have made the system call wrapper a cancellation point. (If we
>>>> implement the simpler getentropy interface, it would not be a
>>>> cancellation point.)
>>>
>>>
>>> Why did you do that?
>>
>> I have to, because it can block indefinitely.
>
> Is it practical to make it a cancellation point if and only if called
> in blocking mode? I suspect that is the behavior basically everyone
> wants. Unlike with file descriptors, blocking/nonblocking is visible
> in the flags passed to the function, so it might not be hard.
The predominant use case is to call getrandom to request blocking (with
flags == 0), without expecting blocking (because the kernel pool is
initialized). This would have to be a cancellation point, only the
obscure GRND_NONBLOCK would not be. So I don't think it's worth the
complexity.
(The current goal is to provide a minimal system call wrapper.)
> (What does Solaris' implementation do? OpenBSD's?)
I don't have access to a Solaris system with a compiler. The manual
page doesn't go into that level of detail.
Does OpenBSD have getrandom? I'm not sure if their cancellation
implementation is comparable, either.
Florian
On Fri, 2016-09-09 at 16:28 +0200, Florian Weimer wrote:
> On 09/09/2016 04:21 PM, Torvald Riegel wrote:
> > On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
> >> I have made the system call wrapper a cancellation point. (If we
> >> implement the simpler getentropy interface, it would not be a
> >> cancellation point.)
> >
> > Why did you do that?
>
> I have to, because it can block indefinitely.
That doesn't mean you have to make the default function a cancellation
point. There are many POSIX functions which can block indefinitely and
which are not required to be cancellation points (eg, rwlocks only *may*
be cancellation points).
Can the system call really block indefinitely, or only for a long time
and (ie, will return eventually)?
> > Even though the system call is new, and thus can't
> > have been used in existing code directly, making it a cancellation point
> > will make all callers cancellation points too. Therefore, for example,
> > we couldn't use it in the implementation of any POSIX functions (that
> > are not cancellation points) in glibc without having to disable and
> > restore the cancellation state around it every time.
>
> The system call definition facility also provides a
> __getrandom_notcancel entry point, which can be called if we don't
> expect blocking. Obviously, this is for internal use only.
>
> > It might be even more convenient to have one wrapper that is a
> > cancellation point and one that is not.
> >
> > Can't we just let cancellation rot in its corner?
>
> No, we have many customers who use it (and this despite the fact that
> the current implementation has a critical race condition).
Usage of it doesn't mean that it has to be the default. Have we made
other syscall wrappers cancellation points in the past (ie, syscalls
that don't already have a matching POSIX function that is specified to
be a cancellation point too)?
I'm worried about people who just want to use the syscall but don't know
that much about POSIX cancellation. They couldn't use the syscall
safely in a library without also being aware of POSIX cancellation, and
I'm concerned that they might just forget to disable cancellation around
the syscall, thus creating resource leaks, deadlocks (eg, cancellation
handler doesn't release locks), etc. If this is primarily a Linux API
currently (ignoring the Solaris case for a while), then marrying it to
POSIX seems wrong.
On 09/09/2016 05:23 PM, Torvald Riegel wrote:
> On Fri, 2016-09-09 at 16:28 +0200, Florian Weimer wrote:
>> On 09/09/2016 04:21 PM, Torvald Riegel wrote:
>>> On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
>>>> I have made the system call wrapper a cancellation point. (If we
>>>> implement the simpler getentropy interface, it would not be a
>>>> cancellation point.)
>>>
>>> Why did you do that?
>>
>> I have to, because it can block indefinitely.
>
> That doesn't mean you have to make the default function a cancellation
> point. There are many POSIX functions which can block indefinitely and
> which are not required to be cancellation points (eg, rwlocks only *may*
> be cancellation points).
>
> Can the system call really block indefinitely, or only for a long time
> and (ie, will return eventually)?
Yes, if the system enters a deadlock condition where the waiting for
randomness prevents it from accumulating additional randomness.
>>> Can't we just let cancellation rot in its corner?
>>
>> No, we have many customers who use it (and this despite the fact that
>> the current implementation has a critical race condition).
>
> Usage of it doesn't mean that it has to be the default.
It's not used by default. Something has to call pthread_cancel.
> Have we made
> other syscall wrappers cancellation points in the past (ie, syscalls
> that don't already have a matching POSIX function that is specified to
> be a cancellation point too)?
I found open_by_handle.
> I'm worried about people who just want to use the syscall but don't know
> that much about POSIX cancellation. They couldn't use the syscall
> safely in a library without also being aware of POSIX cancellation, and
> I'm concerned that they might just forget to disable cancellation around
> the syscall, thus creating resource leaks, deadlocks (eg, cancellation
> handler doesn't release locks), etc. If this is primarily a Linux API
> currently (ignoring the Solaris case for a while), then marrying it to
> POSIX seems wrong.
If we add getentropy, I suggest that it will not be a cancellation point
(even if it can still block indefinitely).
I looked at quite a few getrandom emulations using /dev/urandom, and not
one of them was cancellation-aware (it leaked the file descriptor on
cancellation, for example). Based on that, I really doubt getrandom
would introduce an unexpected cancellation point that causes actual
problems.
Florian
On Mon, 2016-09-12 at 09:25 +0200, Florian Weimer wrote:
> On 09/09/2016 05:23 PM, Torvald Riegel wrote:
> > On Fri, 2016-09-09 at 16:28 +0200, Florian Weimer wrote:
> >> On 09/09/2016 04:21 PM, Torvald Riegel wrote:
> >>> Can't we just let cancellation rot in its corner?
> >>
> >> No, we have many customers who use it (and this despite the fact that
> >> the current implementation has a critical race condition).
> >
> > Usage of it doesn't mean that it has to be the default.
>
> It's not used by default. Something has to call pthread_cancel.
I do mean the other side. That is, in all the code that may see a
cancellation request.
> > Have we made
> > other syscall wrappers cancellation points in the past (ie, syscalls
> > that don't already have a matching POSIX function that is specified to
> > be a cancellation point too)?
>
> I found open_by_handle.
OK, though that's much like open(), which is a cancellation point, so
making the syscall a cancellation point too would make sense.
The pseudo-RNG functions are not cancellation points.
> > I'm worried about people who just want to use the syscall but don't know
> > that much about POSIX cancellation. They couldn't use the syscall
> > safely in a library without also being aware of POSIX cancellation, and
> > I'm concerned that they might just forget to disable cancellation around
> > the syscall, thus creating resource leaks, deadlocks (eg, cancellation
> > handler doesn't release locks), etc. If this is primarily a Linux API
> > currently (ignoring the Solaris case for a while), then marrying it to
> > POSIX seems wrong.
>
> If we add getentropy, I suggest that it will not be a cancellation point
> (even if it can still block indefinitely).
Can you elaborate on your reasoning?
> I looked at quite a few getrandom emulations using /dev/urandom, and not
> one of them was cancellation-aware (it leaked the file descriptor on
> cancellation, for example). Based on that, I really doubt getrandom
> would introduce an unexpected cancellation point that causes actual
> problems.
Interesting, thanks. That might be one interpretation of the situation
(ie, that users know that they don't have to worry about cancellation
requests concurrent or pending while getting a random number).
However, it might also mean that what I worry about is actually
realistic (ie, that user code should be cancellation-aware but isn't).
On 09/12/2016 11:40 AM, Torvald Riegel wrote:
> On Mon, 2016-09-12 at 09:25 +0200, Florian Weimer wrote:
>> On 09/09/2016 05:23 PM, Torvald Riegel wrote:
>>> On Fri, 2016-09-09 at 16:28 +0200, Florian Weimer wrote:
>>>> On 09/09/2016 04:21 PM, Torvald Riegel wrote:
>>>>> Can't we just let cancellation rot in its corner?
>>>>
>>>> No, we have many customers who use it (and this despite the fact that
>>>> the current implementation has a critical race condition).
>>>
>>> Usage of it doesn't mean that it has to be the default.
>>
>> It's not used by default. Something has to call pthread_cancel.
>
> I do mean the other side. That is, in all the code that may see a
> cancellation request.
Only very little code supports that. Even in glibc, we have many parts
that have cancellation points, but do not install cancellation handlers
to clean up resources.
Changing that probably needs some form of tools support. Doing this
right is mostly a matter of looking at the call graph.
>>> Have we made
>>> other syscall wrappers cancellation points in the past (ie, syscalls
>>> that don't already have a matching POSIX function that is specified to
>>> be a cancellation point too)?
>>
>> I found open_by_handle.
>
> OK, though that's much like open(), which is a cancellation point, so
> making the syscall a cancellation point too would make sense.
> The pseudo-RNG functions are not cancellation points.
I see getrandom more like read, or ioctl. It's a matter of perspective.
arc4random would be more like a PRNG (can't fail, no short reads, no
blocking (although this part may be difficult), no cancellation point,
hundreds of megabytes per second throughput on current machines).
getentropy is somewhere in the middle.
>>> I'm worried about people who just want to use the syscall but don't know
>>> that much about POSIX cancellation. They couldn't use the syscall
>>> safely in a library without also being aware of POSIX cancellation, and
>>> I'm concerned that they might just forget to disable cancellation around
>>> the syscall, thus creating resource leaks, deadlocks (eg, cancellation
>>> handler doesn't release locks), etc. If this is primarily a Linux API
>>> currently (ignoring the Solaris case for a while), then marrying it to
>>> POSIX seems wrong.
>>
>> If we add getentropy, I suggest that it will not be a cancellation point
>> (even if it can still block indefinitely).
>
> Can you elaborate on your reasoning?
getentropy is supposed to be the simple interface cryptographic
libraries should use to obtain a seed for their own PRNG. Not making it
a cancellation point is part of keeping the interface simple.
>> I looked at quite a few getrandom emulations using /dev/urandom, and not
>> one of them was cancellation-aware (it leaked the file descriptor on
>> cancellation, for example). Based on that, I really doubt getrandom
>> would introduce an unexpected cancellation point that causes actual
>> problems.
>
> Interesting, thanks. That might be one interpretation of the situation
> (ie, that users know that they don't have to worry about cancellation
> requests concurrent or pending while getting a random number).
> However, it might also mean that what I worry about is actually
> realistic (ie, that user code should be cancellation-aware but isn't).
It matters only if the code can run in a thread which is canceled.
I have to admit that references to pthread_cancel are more widespread
than I assumed. Yet we see few bug reports related to cancellation,
despite that both glibc's implementation and library awareness of
cancellation leave a lot to be desired.
Anyway, this is probably material for another thread. :)
Florian
On Mon, 2016-09-12 at 09:25 +0200, Florian Weimer wrote:
> On 09/09/2016 05:23 PM, Torvald Riegel wrote:
> > On Fri, 2016-09-09 at 16:28 +0200, Florian Weimer wrote:
> >> On 09/09/2016 04:21 PM, Torvald Riegel wrote:
> >>> On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
> >>>> I have made the system call wrapper a cancellation point. (If we
> >>>> implement the simpler getentropy interface, it would not be a
> >>>> cancellation point.)
> >>>
> >>> Why did you do that?
> >>
> >> I have to, because it can block indefinitely.
> >
> > That doesn't mean you have to make the default function a cancellation
> > point. There are many POSIX functions which can block indefinitely and
> > which are not required to be cancellation points (eg, rwlocks only *may*
> > be cancellation points).
> >
> > Can the system call really block indefinitely, or only for a long time
> > and (ie, will return eventually)?
>
> Yes, if the system enters a deadlock condition where the waiting for
> randomness prevents it from accumulating additional randomness.
Is it known that this can happen under certain situations, or is it just
unproven that this cannot happen?
I would guess that it cannot happen, because at some point, waiting
probably needs polling or can be implemented with polling. Is there a
concern that polling and the timing differences that arise when doing so
do not feature enough randomness to be able to eventually provide enough
random data?
On 09/23/2016 11:44 AM, Torvald Riegel wrote:
> On Mon, 2016-09-12 at 09:25 +0200, Florian Weimer wrote:
>> On 09/09/2016 05:23 PM, Torvald Riegel wrote:
>>> On Fri, 2016-09-09 at 16:28 +0200, Florian Weimer wrote:
>>>> On 09/09/2016 04:21 PM, Torvald Riegel wrote:
>>>>> On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
>>>>>> I have made the system call wrapper a cancellation point. (If we
>>>>>> implement the simpler getentropy interface, it would not be a
>>>>>> cancellation point.)
>>>>>
>>>>> Why did you do that?
>>>>
>>>> I have to, because it can block indefinitely.
>>>
>>> That doesn't mean you have to make the default function a cancellation
>>> point. There are many POSIX functions which can block indefinitely and
>>> which are not required to be cancellation points (eg, rwlocks only *may*
>>> be cancellation points).
>>>
>>> Can the system call really block indefinitely, or only for a long time
>>> and (ie, will return eventually)?
>>
>> Yes, if the system enters a deadlock condition where the waiting for
>> randomness prevents it from accumulating additional randomness.
>
> Is it known that this can happen under certain situations, or is it just
> unproven that this cannot happen?
I don't think anyone has seen this yet because they use /dev/urandom,
which does not block when the pool is initialized. The request we got
for getrandom was specifically to enable blocking until pool initialization.
Delayed initialization of the pool does happen. For example, I'm
currently logged into an aarch64 system (whose root file system is on an
SSD) which initialized the pool 37 seconds after starting the kernel, at
which point many userland processes had already been started.
(I'm not sure if it matters for the cancellation point discussion
whether the blocking is truly indefinite, or just impracticably long.)
> I would guess that it cannot happen, because at some point, waiting
> probably needs polling or can be implemented with polling. Is there a
> concern that polling and the timing differences that arise when doing so
> do not feature enough randomness to be able to eventually provide enough
> random data?
Time itself is not necessarily used as a randomness source in the kernel.
Thanks,
Florian
On 09/09/2016 10:21 AM, Torvald Riegel wrote:
> On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
>> I have made the system call wrapper a cancellation point. (If we
>> implement the simpler getentropy interface, it would not be a
>> cancellation point.)
>
> Why did you do that? Even though the system call is new, and thus can't
> have been used in existing code directly, making it a cancellation point
> will make all callers cancellation points too. Therefore, for example,
> we couldn't use it in the implementation of any POSIX functions (that
> are not cancellation points) in glibc without having to disable and
> restore the cancellation state around it every time.
> It might be even more convenient to have one wrapper that is a
> cancellation point and one that is not.
>
> Can't we just let cancellation rot in its corner?
I will second what Florian has suggested, I also think getrandom() should
be a cancellation point.
* I see getrandom() as more like a read() and read() is cancellation point.
The argument that the functions can block indefinitely makes them prime
candidates for cancellation. The fact that rwlock-related, or in fact all
pthread-related functions are not cancellation points is not a great
comparison because threads are sufficiently complex that their cancellation
is generally a combination of explicit cancellation checks and complex cancel
handlers rather than pthread routines being cancellation points (too simple a model).
* Application authors that do not want to think about cancellation should disable it
e.g. pthread_setcancelstate (PTHREAD_CANCEL_DISABLE, NULL), and libraries that
enable it should only cancel _their own_ threads which are cancellation aware.
This speaks broadly to Torvald's concerns about user code being canceled by
accident. I would much rather see glibc set cancellation to disabled by default
instead of the POSIX mandated 'enabled and deferred' by default, and document
the difference from POSIX (the gun should be unloaded and in a locked box).
* Internal use of a non-cancellating version is done easily by calling
__getrandom_nocancel() (As noted by Florian).
* It is true that making it a cancellation point will make all callers
a cancellation point, but that's already true for much more common
cancellation points like read() and write() and printf().
* Making it a cancellation point that is dependent on input flags e.g.
non-blocking, will complicate the simple wrapper and make it more
difficult to explain to users since it can no longer go on a simple
list of cancellation points.
* Sun OS 5.11 (Open Indiana) passes the glibc tst-getrandom regression test.
The getrandom() call in OI is _not_ a cancellation point, and verified with
a test case. It appears to be a very small wrapper around a kernel getrandom
syscall (disassembly of libc.so.1). Even if I think this is wrong, it means
we might see problems porting from Solaris to Linux, but only in the intersection
of applications that use cancellation and getrandom. I also note that getentropy
seems built upon getrandom (disassembly shows that).
On 10/03/2016 07:51 PM, Carlos O'Donell wrote:
> On 09/09/2016 10:21 AM, Torvald Riegel wrote:
>> On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
>>> I have made the system call wrapper a cancellation point. (If we
>>> implement the simpler getentropy interface, it would not be a
>>> cancellation point.)
>>
>> Why did you do that? Even though the system call is new, and thus can't
>> have been used in existing code directly, making it a cancellation point
>> will make all callers cancellation points too. Therefore, for example,
>> we couldn't use it in the implementation of any POSIX functions (that
>> are not cancellation points) in glibc without having to disable and
>> restore the cancellation state around it every time.
>> It might be even more convenient to have one wrapper that is a
>> cancellation point and one that is not.
>>
>> Can't we just let cancellation rot in its corner?
>
> I will second what Florian has suggested, I also think getrandom() should
> be a cancellation point.
Thanks.
Torvald, do these arguments convince you?
Are there any remaining objects about the system call as posted?
Thanks,
Florian
On 04/10/2016 05:02, Florian Weimer wrote:
> On 10/03/2016 07:51 PM, Carlos O'Donell wrote:
>> On 09/09/2016 10:21 AM, Torvald Riegel wrote:
>>> On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
>>>> I have made the system call wrapper a cancellation point. (If we
>>>> implement the simpler getentropy interface, it would not be a
>>>> cancellation point.)
>>>
>>> Why did you do that? Even though the system call is new, and thus can't
>>> have been used in existing code directly, making it a cancellation point
>>> will make all callers cancellation points too. Therefore, for example,
>>> we couldn't use it in the implementation of any POSIX functions (that
>>> are not cancellation points) in glibc without having to disable and
>>> restore the cancellation state around it every time.
>>> It might be even more convenient to have one wrapper that is a
>>> cancellation point and one that is not.
>>>
>>> Can't we just let cancellation rot in its corner?
>>
>> I will second what Florian has suggested, I also think getrandom() should
>> be a cancellation point.
I also see a better alternative to set getrandom as a cancellation point,
it aligns with latest linux oriented syscalls that might block ({p}pool,
sync_file_range, epoll_{p}wait).
On Mon, 2016-10-03 at 13:51 -0400, Carlos O'Donell wrote:
> On 09/09/2016 10:21 AM, Torvald Riegel wrote:
> > On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
> >> I have made the system call wrapper a cancellation point. (If we
> >> implement the simpler getentropy interface, it would not be a
> >> cancellation point.)
> >
> > Why did you do that? Even though the system call is new, and thus can't
> > have been used in existing code directly, making it a cancellation point
> > will make all callers cancellation points too. Therefore, for example,
> > we couldn't use it in the implementation of any POSIX functions (that
> > are not cancellation points) in glibc without having to disable and
> > restore the cancellation state around it every time.
> > It might be even more convenient to have one wrapper that is a
> > cancellation point and one that is not.
> >
> > Can't we just let cancellation rot in its corner?
>
> I will second what Florian has suggested, I also think getrandom() should
> be a cancellation point.
>
> * I see getrandom() as more like a read() and read() is cancellation point.
> The argument that the functions can block indefinitely makes them prime
> candidates for cancellation.
I agree that cancellation can make sense for them (assuming one wants
cancellation ;) ).
> The fact that rwlock-related, or in fact all
> pthread-related functions are not cancellation points
Some are cancellation points.
> is not a great
> comparison because threads are sufficiently complex that their cancellation
> is generally a combination of explicit cancellation checks and complex cancel
> handlers rather than pthread routines being cancellation points (too simple a model).
I don't quite understand this part of the sentence.
> * Application authors that do not want to think about cancellation should disable it
> e.g. pthread_setcancelstate (PTHREAD_CANCEL_DISABLE, NULL), and libraries that
> enable it should only cancel _their own_ threads which are cancellation aware.
> This speaks broadly to Torvald's concerns about user code being canceled by
> accident. I would much rather see glibc set cancellation to disabled by default
> instead of the POSIX mandated 'enabled and deferred' by default, and document
> the difference from POSIX (the gun should be unloaded and in a locked box).
That would be good, but I guess we can't do it without potentially
breaking existing programs?
> * Internal use of a non-cancellating version is done easily by calling
> __getrandom_nocancel() (As noted by Florian).
I'm not that much concerned about internal use. It was just an example.
The external uses concern me.
> * Making it a cancellation point that is dependent on input flags e.g.
> non-blocking, will complicate the simple wrapper and make it more
> difficult to explain to users since it can no longer go on a simple
> list of cancellation points.
While that is true, I think it rather hides the complexity. Who really
looks at the list of cancellation points?
From a birds-eye view, I think it would have been better if users would
explicitly enable cancellation points, instead of vice versa (see the
default of enabled+deferred you mention). Like we do for timeouts, for
example.
However, POSIX doesn't specify that. I'd prefer if we would not add
more of that in the form of more cancellation points by default, but I
also can't argue that what you prefer would be inconsistent with POSIX.
Given that, and that a majority seems to share your opinion, I guess we
have consensus.
On Tue, Oct 4, 2016 at 12:05 PM, Torvald Riegel <triegel@redhat.com> wrote:
>
> From a birds-eye view, I think it would have been better if users would
> explicitly enable cancellation points, instead of vice versa (see the
> default of enabled+deferred you mention). Like we do for timeouts, for
> example.
> However, POSIX doesn't specify that. I'd prefer if we would not add
> more of that in the form of more cancellation points by default, but I
> also can't argue that what you prefer would be inconsistent with POSIX.
>
> Given that, and that a majority seems to share your opinion, I guess we
> have consensus.
I'll take getrandom that's a cancellation point over no getrandom at
all, but I think there are very strong reasons why getrandom,
specifically, should *not* be a cancellation point, whether or not it
can block.
Fundamentally the problem I have with it is that getrandom must always
succeed; applications cannot realistically be expected to recover from
failure. In fact, under sufficiently nasty adversarial conditions it
might not be acceptable for getrandom to *crash* -- look up the
concept of a "bomb oracle". The deviations from ideal behavior that
already exist (might block, might return fewer bytes than requested)
may already render it unfit for purpose! Adding cancellation into the
mix is just another headache and a reason for people to either not use
getrandom at all, or continue manually open-coding the syscall.
zw
On 10/04/2016 10:30 PM, Zack Weinberg wrote:
> Fundamentally the problem I have with it is that getrandom must always
> succeed; applications cannot realistically be expected to recover from
> failure.
The current patch is the consensus which emerged after the first
patches. In particular, you wrote a while back, “I think it would be
better to expose getrandom() as a completely unadorned syscall.”
As a result of this community-support decision, ENOSYS due to lack of
kernel support is always a possibility. I think this was pretty clear
at the time, when the emulation based on device nodes was rejected.
Florian
On Wed, Oct 5, 2016 at 3:18 AM, Florian Weimer <fweimer@redhat.com> wrote:
> On 10/04/2016 10:30 PM, Zack Weinberg wrote:
>> Fundamentally the problem I have with it is that getrandom must always
>> succeed; applications cannot realistically be expected to recover from
>> failure.
>
> The current patch is the consensus which emerged after the first patches.
> In particular, you wrote a while back, “I think it would be better to expose
> getrandom() as a completely unadorned syscall.”
But note that an unadorned syscall is *not* a cancellation point.
> As a result of this community-support decision, ENOSYS due to lack of kernel
> support is always a possibility. I think this was pretty clear at the time,
> when the emulation based on device nodes was rejected.
I would have supported the emulation based on device nodes if I had
thought there was any way to make it reliable enough that it would not
cause more problems than it solved.
zw
On 09/12/2016 09:25 AM, Florian Weimer wrote:
> On 09/09/2016 05:23 PM, Torvald Riegel wrote:
>> On Fri, 2016-09-09 at 16:28 +0200, Florian Weimer wrote:
>>> On 09/09/2016 04:21 PM, Torvald Riegel wrote:
>>>> On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
>>>>> I have made the system call wrapper a cancellation point. (If we
>>>>> implement the simpler getentropy interface, it would not be a
>>>>> cancellation point.)
>>>>
>>>> Why did you do that?
>>>
>>> I have to, because it can block indefinitely.
>>
>> That doesn't mean you have to make the default function a cancellation
>> point. There are many POSIX functions which can block indefinitely and
>> which are not required to be cancellation points (eg, rwlocks only *may*
>> be cancellation points).
>>
>> Can the system call really block indefinitely, or only for a long time
>> and (ie, will return eventually)?
>
> Yes, if the system enters a deadlock condition where the waiting for
> randomness prevents it from accumulating additional randomness.
This is what happens here:
<https://bugzilla.redhat.com/show_bug.cgi?id=1383060>
systemd will eventually kill the blocked process and the boot continues,
but all network services will be missing.
Florian
On Wed, Oct 12, 2016 at 11:58 AM, Florian Weimer <fweimer@redhat.com> wrote:
>>> Can the system call really block indefinitely, or only for a long time
>>> and (ie, will return eventually)?
>>
>> Yes, if the system enters a deadlock condition where the waiting for
>> randomness prevents it from accumulating additional randomness.
>
> This is what happens here:
>
> <https://bugzilla.redhat.com/show_bug.cgi?id=1383060>
... How about we make the kernel not start userspace at all until the
RNG is fully operational?
zw
On Wed, 2016-10-12 at 17:58 +0200, Florian Weimer wrote:
> On 09/12/2016 09:25 AM, Florian Weimer wrote:
> > On 09/09/2016 05:23 PM, Torvald Riegel wrote:
> >> On Fri, 2016-09-09 at 16:28 +0200, Florian Weimer wrote:
> >>> On 09/09/2016 04:21 PM, Torvald Riegel wrote:
> >>>> On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
> >>>>> I have made the system call wrapper a cancellation point. (If we
> >>>>> implement the simpler getentropy interface, it would not be a
> >>>>> cancellation point.)
> >>>>
> >>>> Why did you do that?
> >>>
> >>> I have to, because it can block indefinitely.
> >>
> >> That doesn't mean you have to make the default function a cancellation
> >> point. There are many POSIX functions which can block indefinitely and
> >> which are not required to be cancellation points (eg, rwlocks only *may*
> >> be cancellation points).
> >>
> >> Can the system call really block indefinitely, or only for a long time
> >> and (ie, will return eventually)?
> >
> > Yes, if the system enters a deadlock condition where the waiting for
> > randomness prevents it from accumulating additional randomness.
>
> This is what happens here:
>
> <https://bugzilla.redhat.com/show_bug.cgi?id=1383060>
>
> systemd will eventually kill the blocked process and the boot continues,
> but all network services will be missing.
I don't see how cancellation would be the best solution for this
problem. It could be considered *a* solution, but for it to work, the
program (or python in this case) needs to
(1) be aware of the blocking behavior, and potential absence of
getrandom initialization
(2) have a fallback plan for when getrandom() fails
(3) run more than a single thread just to run the cancellation
(4) synchronize between the thread that executes getrandom() and the
cancelling thread so that pthread_cancel can cancel only the respective
getrandom() call; for example, the getrandom thread could set/unset a
flag in a critical section, and the cancelling thread needs to check the
flag and cancel in a critical section protected by the same lock.
All solutions have to be aware of (1).
Maybe doing the fallback plan from (2) is the right solution. This is
what they seem to have done in the BZ you cited.
Having a getrandom call that can time out would be easier for programs
to use than cancellation. It would still require picking the right
timeout (ie, less than systemd's), but that's somethign the
cancellation-based solution would have to do too.
Cancellation would be more a more flexible means than a timeout if
systemd would announce to the program that it will get killed in 5s or
something like that, and then the program could try to hurry up and
cancel the getrandom call. But that seems like a complex and rather
rare use case, and thus wouldn't suggest to me that a cancellable
getrandom should be the default.
Instead, using a provided getrandom_cancellable wrapper seems like the
right thing, and trivial compared to all the other complexity the
program has to deal with.
On 10/17/2016 02:54 PM, Torvald Riegel wrote:
> On Wed, 2016-10-12 at 17:58 +0200, Florian Weimer wrote:
>> On 09/12/2016 09:25 AM, Florian Weimer wrote:
>>> On 09/09/2016 05:23 PM, Torvald Riegel wrote:
>>>> On Fri, 2016-09-09 at 16:28 +0200, Florian Weimer wrote:
>>>>> On 09/09/2016 04:21 PM, Torvald Riegel wrote:
>>>>>> On Thu, 2016-09-08 at 13:44 +0200, Florian Weimer wrote:
>>>>>>> I have made the system call wrapper a cancellation point. (If we
>>>>>>> implement the simpler getentropy interface, it would not be a
>>>>>>> cancellation point.)
>>>>>>
>>>>>> Why did you do that?
>>>>>
>>>>> I have to, because it can block indefinitely.
>>>>
>>>> That doesn't mean you have to make the default function a cancellation
>>>> point. There are many POSIX functions which can block indefinitely and
>>>> which are not required to be cancellation points (eg, rwlocks only *may*
>>>> be cancellation points).
>>>>
>>>> Can the system call really block indefinitely, or only for a long time
>>>> and (ie, will return eventually)?
>>>
>>> Yes, if the system enters a deadlock condition where the waiting for
>>> randomness prevents it from accumulating additional randomness.
>>
>> This is what happens here:
>>
>> <https://bugzilla.redhat.com/show_bug.cgi?id=1383060>
>>
>> systemd will eventually kill the blocked process and the boot continues,
>> but all network services will be missing.
>
> I don't see how cancellation would be the best solution for this
> problem.
You asked if the system call can really block indefinitely, and we now
have empirical evidence that it can happen.
Florian
On 10/12/2016 06:10 PM, Zack Weinberg wrote:
> On Wed, Oct 12, 2016 at 11:58 AM, Florian Weimer <fweimer@redhat.com> wrote:
>>>> Can the system call really block indefinitely, or only for a long time
>>>> and (ie, will return eventually)?
>>>
>>> Yes, if the system enters a deadlock condition where the waiting for
>>> randomness prevents it from accumulating additional randomness.
>>
>> This is what happens here:
>>
>> <https://bugzilla.redhat.com/show_bug.cgi?id=1383060>
>
> ... How about we make the kernel not start userspace at all until the
> RNG is fully operational?
The RNG is initialized from user space, using a saved seed from disk
(which is better than no randomness at all).
Florian
On Mon, Oct 17, 2016 at 9:02 AM, Florian Weimer <fweimer@redhat.com> wrote:
> On 10/12/2016 06:10 PM, Zack Weinberg wrote:
>> ... How about we make the kernel not start userspace at all until the
>> RNG is fully operational?
>
> The RNG is initialized from user space, using a saved seed from disk (which
> is better than no randomness at all).
I know. I'm questioning whether that is still a good plan.
zw
Add getrandom system call and <sys/random.h> header file [BZ #17252]
2016-09-08 Florian Weimer <fweimer@redhat.com>
[BZ #17252]
* stdlib/sys/random.h: New file.
(headers): Add it.
* stdlib/Makefile (routines): Add getrandom.
(tests): Add tst-getrandom.
* stdlib/Versions (GLIBC_2.25): Add __getrandom.
* stdlib/getrandom.c: New file.
* stdlib/tst-getrandom.c: Likewise.
* manual/crypt.texi (Unpredictable Bytes): New section.
* manual/math.texi (Pseudo-Random Numbers): Add cross-reference.
* sysdeps/arm/nacl/libc.abilist: Add __getrandom.
* sysdeps/unix/sysv/linux/aarch64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/alpha/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/arm/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/hppa/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/i386/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/ia64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/microblaze/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/nios2/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist:
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist:
Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc-le.abilist: Likewise.
* sysdeps/unix/sysv/linux/powerpc/powerpc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sh/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx32/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilegx/tilegx64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/tile/tilepro/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/x86_64/64/libc.abilist: Likewise.
* sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist: Likewise.
@@ -40,6 +40,8 @@ Version 2.25
for the Linux quota interface which predates kernel version 2.4.22 has
been removed.
+* The getrandom function and the <sys/random.h> header file have been added.
+
Security related changes:
On ARM EABI (32-bit), generating a backtrace for execution contexts which
@@ -45,6 +45,7 @@ encrypted authentication use normal DES.
* getpass:: Prompting the user for a password.
* crypt:: A one-way function for passwords.
* DES Encryption:: Routines for DES encryption.
+* Unpredictable Bytes:: Randomness for cryptography purposes.
@end menu
@node Legal Problems
@@ -428,3 +429,69 @@ each byte.
The @code{ecb_crypt}, @code{cbc_crypt}, and @code{des_setparity}
functions and their accompanying macros are all defined in the header
@file{rpc/des_crypt.h}.
+
+@node Unpredictable Bytes
+@section Generating Unpredictable Bytes
+
+Some cryptographic applications (such as session key generation) need
+unpredictable bytes.
+
+@comment sys/random.h
+@comment GNU
+@deftypefun ssize_t getrandom (void *@var{buffer}, size_t @var{length}, unsigned int @var{flags})
+@safety{@mtsafe{}@assafe{}@acsafe{}}
+
+This function writes @var{length} bytes of random data to the array
+starting at @var{buffer}. On succes, this function returns the number
+of bytes which have been written to the buffer (which can be less than
+@var{length}). On error, @code{-1} is returned, and @code{errno} is
+updated accordingly.
+
+The @code{getrandom} function is declared in the header file
+@file{sys/random.h}. It is a GNU extension.
+
+The following flags are defined for the @var{flags} argument:
+
+@table @code
+@item GRND_RANDOM
+Use the blocking pool instead of the non-blocking pool to obtain
+randomness. By default, the non-blocking pool is used. The blocking
+pool corresponds to @file{/dev/random}, and the non-blocking pool to
+@file{/dev/urandom}.
+
+@item GRND_NONBLOCK
+Instead of blocking, return to the caller immediately if no data is
+available.
+@end table
+
+The @code{getrandom} function is a cancellation point.
+
+Even access to the non-blocking pool can block if the system has just
+booted and the pool has not yet been initialized.
+
+The @code{getrandom} function can fail with several errors, some of
+which are listed below. In addition, the function may not fill the
+buffer completely and return a value less than @var{length}.
+
+@table @code
+@item ENOSYS
+The kernel does not implement the @code{getrandom} system call.
+
+@item EAGAIN
+No random data was available and @code{GRND_NONBLOCK} was specified in
+@var{flags}.
+
+@item EFAULT
+The the combination of @var{buffer} and @var{length} arguments specifies
+an invalid memory range.
+
+@item EINTR
+The system call was interrupted. During the system boot process, before
+the kernel randomness pool is initialized, this can happen even if
+@var{flags} is zero.
+
+@item EINVAL
+The @var{flags} argument contains an invalid combination of flags.
+@end table
+
+@end deftypefun
@@ -1414,7 +1414,8 @@ is convenient when you are debugging a program, but it is unhelpful if
you want the program to behave unpredictably. If you want a different
pseudo-random series each time your program runs, you must specify a
different seed each time. For ordinary purposes, basing the seed on the
-current time works well.
+current time works well. For random numbers in cryptography,
+@pxref{Unpredictable Bytes}.
You can obtain repeatable sequences of numbers on a particular machine type
by specifying the same initial seed value for the random number
@@ -28,7 +28,7 @@ headers := stdlib.h bits/stdlib.h bits/stdlib-ldbl.h bits/stdlib-float.h \
errno.h sys/errno.h bits/errno.h \
ucontext.h sys/ucontext.h \
alloca.h fmtmsg.h \
- bits/stdlib-bsearch.h
+ bits/stdlib-bsearch.h sys/random.h
routines := \
atof atoi atol atoll \
@@ -45,7 +45,7 @@ routines := \
srand48 seed48 lcong48 \
drand48_r erand48_r lrand48_r nrand48_r mrand48_r jrand48_r \
srand48_r seed48_r lcong48_r \
- drand48-iter \
+ drand48-iter getrandom \
strtol strtoul strtoll strtoull \
strtol_l strtoul_l strtoll_l strtoull_l \
strtof strtod strtold \
@@ -77,7 +77,7 @@ tests := tst-strtol tst-strtod testmb testrand testsort testdiv \
tst-tininess tst-strtod-underflow tst-tls-atexit \
tst-setcontext3 tst-tls-atexit-nodelete \
tst-strtol-locale tst-strtod-nan-locale tst-strfmon_l \
- tst-quick_exit tst-thread-quick_exit
+ tst-quick_exit tst-thread-quick_exit tst-getrandom
tests-static := tst-secure-getenv
ifeq ($(have-cxx-thread_local),yes)
CFLAGS-tst-quick_exit.o = -std=c++11
@@ -112,6 +112,9 @@ libc {
GLIBC_2.24 {
quick_exit;
}
+ GLIBC_2.25 {
+ __getrandom;
+ }
GLIBC_PRIVATE {
# functions which have an additional interface since they are
# are cancelable.
new file mode 100644
@@ -0,0 +1,31 @@
+/* Stub for getrandom.
+ Copyright (C) 2016 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <errno.h>
+#include <sys/random.h>
+
+/* Write LENGTH bytes of randomness starting at BUFFER. Returns the
+ number of bytes written, or -1 on error. */
+ssize_t
+__getrandom (void *buffer, size_t length, unsigned int flags)
+{
+ __set_errno (ENOSYS);
+ return -1;
+}
+
+stub_warning (__getrandom)
new file mode 100644
@@ -0,0 +1,35 @@
+/* Interfaces for obtaining random bytes.
+ Copyright (C) 2016 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef _SYS_RANDOM_H
+#define _SYS_RANDOM_H 1
+
+/* Flags for use with getrandom. */
+# define GRND_NONBLOCK 1
+# define GRND_RANDOM 2
+
+/* Write LENGTH bytes of randomness starting at BUFFER. Returns the
+ number of bytes written, or -1 on error. */
+ssize_t __getrandom (void *__buffer, size_t __length, unsigned int __flags)
+ __wur;
+
+/* Prevent accidental interposition of the getrandom symbol. */
+#define getrandom(buffer, length, flags) \
+ (0 + __getrandom (buffer, length, flags))
+
+#endif /* _SYS_RANDOM_H */
new file mode 100644
@@ -0,0 +1,166 @@
+/* Tests for the getrandom function.
+ Copyright (C) 2016 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <errno.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <string.h>
+#include <sys/random.h>
+
+/* Set to true if any errors are encountered. */
+static bool errors;
+
+/* Test getrandom with a single buffer length. */
+static void
+test_length (char *buffer, int length, unsigned flags)
+{
+ memset (buffer, 0, length);
+ strcpy (buffer + length, "123");
+ ssize_t ret = getrandom (buffer, length, flags);
+ if (ret < 0)
+ {
+ if (!((flags & GRND_RANDOM)
+ && (flags & GRND_NONBLOCK)
+ && errno != EAGAIN))
+ {
+ printf ("error: getrandom (%d, 0x%x): %m\n", length, flags);
+ errors = true;
+ }
+ }
+ if (ret != length)
+ {
+ if (flags & GRND_RANDOM)
+ {
+ if (ret == 0 || ret > length)
+ {
+ printf ("error: getrandom (%d, 0x%x) returned %zd\n",
+ length, flags, ret);
+ errors = true;
+ }
+ }
+ else
+ {
+ printf ("error: getrandom (%d, 0x%x) returned %zd\n",
+ length, flags, ret);
+ errors = true;
+ }
+ }
+ if (length >= 7)
+ {
+ /* One spurious test failure in 2**56 is sufficiently
+ unlikely. */
+ int non_null = 0;
+ for (int i = 0; i < length; ++i)
+ non_null += buffer[i] != 0;
+ if (non_null == 0)
+ {
+ printf ("error: getrandom (%d, 0x%x) returned all-zero bytes\n",
+ length, flags);
+ errors = true;
+ }
+ }
+ if (memcmp (buffer + length, "123", 4) != 0)
+ {
+ printf ("error: getrandom (%d, 0x%x) wrote spurious bytes\n",
+ length, flags);
+ errors = true;
+ }
+}
+
+/* Call getrandom repeatedly to fill the buffer. */
+static bool
+getrandom_full (char *buffer, int length, unsigned flags)
+{
+ char *end = buffer + length;
+ while (buffer < end)
+ {
+ ssize_t ret = getrandom (buffer, end - buffer, flags);
+ if (ret < 0)
+ {
+ printf ("error: getrandom (%d, 0x%x): %m\n", length, flags);
+ errors = true;
+ return false;
+ }
+ buffer += ret;
+ }
+
+ return true;
+}
+
+static void
+test_flags (unsigned flags)
+{
+ /* Test various lengths, but only for !GRND_RANDOM, to conserve
+ entropy. */
+ {
+ enum { max_length = 300 };
+ char buffer[max_length + 4];
+ if (flags & GRND_RANDOM)
+ test_length (buffer, 0, flags);
+ else
+ {
+ for (int length = 0; length <= 9; ++length)
+ test_length (buffer, length, flags);
+ test_length (buffer, 16, flags);
+ test_length (buffer, max_length, flags);
+ }
+ }
+
+ /* Test that getrandom returns different data. */
+ if (!(flags & GRND_NONBLOCK))
+ {
+ char buffer1[8];
+ memset (buffer1, 0, sizeof (buffer1));
+
+ char buffer2[8];
+ memset (buffer2, 0, sizeof (buffer2));
+
+ if (getrandom_full (buffer1, sizeof (buffer1), flags)
+ && getrandom_full (buffer1, sizeof (buffer1), flags))
+ {
+ if (memcmp (buffer1, buffer2, sizeof (buffer1)) == 0)
+ {
+ printf ("error: getrandom returns constant value\n");
+ errors = true;
+ }
+ }
+ }
+}
+
+static int
+do_test (void)
+{
+ /* Check if getrandom is not supported by this system. */
+ if (getrandom (NULL, 0, 0) == -1 && errno == ENOSYS)
+ return 77;
+
+ for (int use_random = 0; use_random < 2; ++use_random)
+ for (int use_nonblock = 0; use_nonblock < 2; ++use_nonblock)
+ {
+ int flags = 0;
+ if (use_random)
+ flags |= GRND_RANDOM;
+ if (use_nonblock)
+ flags |= GRND_NONBLOCK;
+ test_flags (flags);
+ }
+ return errors;
+}
+
+#define TEST_FUNCTION do_test ()
+#include "../test-skeleton.c"
@@ -1843,6 +1843,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.25 gnu_dev_major F
GLIBC_2.25 gnu_dev_makedev F
GLIBC_2.25 gnu_dev_minor F
@@ -100,3 +100,4 @@ utimes - utimes i:sp __utimes utimes
vhangup - vhangup i:i vhangup
write - write Ci:ibn __libc_write __write write
writev - writev Ci:ipi __writev writev
+getrandom - getrandom Ci:bni __getrandom
@@ -2090,3 +2090,4 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
@@ -2001,6 +2001,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -91,6 +91,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.4 GLIBC_2.4 A
GLIBC_2.4 _Exit F
GLIBC_2.4 _IO_2_1_stderr_ D 0xa0
@@ -1855,6 +1855,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -2013,6 +2013,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -1877,6 +1877,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -92,6 +92,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.4 GLIBC_2.4 A
GLIBC_2.4 _Exit F
GLIBC_2.4 _IO_2_1_stderr_ D 0x98
@@ -1969,6 +1969,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -2090,3 +2090,4 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
@@ -1944,6 +1944,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -1942,6 +1942,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -1940,6 +1940,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -1935,6 +1935,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -2131,3 +2131,4 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
@@ -1973,6 +1973,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -1978,6 +1978,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -2178,3 +2178,4 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
@@ -92,6 +92,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 _Exit F
GLIBC_2.3 _IO_2_1_stderr_ D 0xe0
@@ -1973,6 +1973,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -1874,6 +1874,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -1859,6 +1859,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -1965,6 +1965,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -1903,6 +1903,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -2097,3 +2097,4 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
@@ -2097,3 +2097,4 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
@@ -2097,3 +2097,4 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
@@ -1854,6 +1854,7 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F
GLIBC_2.3 GLIBC_2.3 A
GLIBC_2.3 __ctype_b_loc F
GLIBC_2.3 __ctype_tolower_loc F
@@ -2097,3 +2097,4 @@ GLIBC_2.23 fts64_set F
GLIBC_2.24 GLIBC_2.24 A
GLIBC_2.24 quick_exit F
GLIBC_2.25 GLIBC_2.25 A
+GLIBC_2.25 __getrandom F