From patchwork Mon Jan 25 20:09:24 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Weimer X-Patchwork-Id: 10558 Received: (qmail 15695 invoked by alias); 25 Jan 2016 20:09:36 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 15575 invoked by uid 89); 25 Jan 2016 20:09:34 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-0.9 required=5.0 tests=BAYES_00, KAM_LAZY_DOMAIN_SECURITY, RCVD_IN_DNSWL_NONE, RP_MATCHES_RCVD autolearn=no version=3.3.2 spammy=487, H*MI:mid, H*M:mid, prime X-HELO: albireo.enyo.de From: Florian Weimer To: Paul Eggert Cc: Florian Weimer , GNU C Library , Adhemerval Zanella Subject: Re: [PATCH] Improve check against integer wraparound in hcreate_r [BZ #18240] References: <56A210C4.80609@redhat.com> <56A42D78.1030506@cs.ucla.edu> Date: Mon, 25 Jan 2016 21:09:24 +0100 In-Reply-To: <56A42D78.1030506@cs.ucla.edu> (Paul Eggert's message of "Sat, 23 Jan 2016 17:48:40 -0800") Message-ID: <877fixs9or.fsf@mid.deneb.enyo.de> MIME-Version: 1.0 * Paul Eggert: > Florian Weimer wrote: > >> - if (nel >= SIZE_MAX / sizeof (_ENTRY)) >> + /* This limit is sufficient to avoid unsigned wraparound below, >> + possibly after truncation to unsigned int. (struct hsearch_data >> + is part of the public API and uses usigned ints.) */ >> + if (nel >= INT_MAX / sizeof (_ENTRY)) > > This patch doesn't look right. nel should be bounded by UINT_MAX - 2, > not by INT_MAX / sizeof (anything). (Not by UINT_MAX, since the code > computes nel + 1 later; and not by UINT_MAX - 1 since that cannot be > prime.) Furthermore, calloc will check for size overflow on > multiplication so hcreate_r need not worry about dividing by sizeof > (anything). Also, "unsigned" is misspelled in the comment. > > How about something like the attached (untested) patch instead? Fair enough. isprime needs to be fixed as well, like this. Adhemerval, do we still have time to fix this? diff --git a/misc/hsearch_r.c b/misc/hsearch_r.c index 7bc04cf..c73d3ed 100644 --- a/misc/hsearch_r.c +++ b/misc/hsearch_r.c @@ -48,7 +48,7 @@ isprime (unsigned int number) /* no even number will be passed */ unsigned int div = 3; - while (div * div < number && number % div != 0) + while (div * (unsigned long long) div < number && number % div != 0) div += 2; return number % div != 0;