From patchwork Thu Oct 19 18:49:28 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Wilco Dijkstra X-Patchwork-Id: 23713 Received: (qmail 56684 invoked by alias); 19 Oct 2017 18:49:35 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 56675 invoked by uid 89); 19 Oct 2017 18:49:35 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-25.2 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.2 spammy=OLD X-HELO: EUR01-HE1-obe.outbound.protection.outlook.com From: Wilco Dijkstra To: DJ Delorie CC: "libc-alpha@sourceware.org" , nd Subject: Re: [PATCH 3/5] Add single-threaded path to _int_free Date: Thu, 19 Oct 2017 18:49:28 +0000 Message-ID: References: (message from Wilco Dijkstra on Thu, 12 Oct 2017 09:35:34 +0000), In-Reply-To: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Wilco.Dijkstra@arm.com; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB6PR0801MB2053; 6:F7auvOI00SchVq2meF4DepPmVk4Sqcnh87t4upBHr1TUMTSpX0OiW8xYbo6PM6g88XvJ0x1jLrdCJD4AoYJhdWEUU7LD4dUo6UFi+89QxBjKfGm1G+DRdqgCK9k1pf3yUp0Vknl8u6ZOFHkYpEhZMB8FOy0yvgquohDJXrdz//1Dwkr6SiV/3nI9aCfbnAfh+nqbXJhA91gB5+rgD4tajw/VK0wHkg4JzUfvLHbYv1WRD9Kacv2gv918eF1SZWdVOSKoiNxDSDr4uMepPtkLjtseQKr/hmL+3Nq1Fa0DQCQfYtaWiCA/K4ujU/6P+fH1UOL7vjbYudc4OruxXVUi3A==; 5:buj+AKZ9bklrEZ6SjlUxWeTt/ytliC5bDL87Xw+JQWbtYi99Hf/Gt1r9ONVrBXgSYScDsyJb424JfODOtW3Tu8fKPRCUHrgnE2/cHXFLMARH7Q4V6P8wSCtXRT51XHYhXLeuc9UrO0x9L9q3qFSXSQ==; 24:znynO2YUa0U9kn6f35990tz+4Ufhxjc5Xtj5kLfhU+MFMvAwZmbvUpMJnBcZiREY4+0e1csG4DxsZAg/w/YcwrNs1t6dX1XCB+NPBuRPRgQ=; 7:m+zKQ+GTvzpISaWkFXxrvMmpRoEks8PN2po4SUAmFX7mbVw6PdrLWyDPOpuKGCeiw7050TMpbVXcGVAtF1KxN3QL2aEJT4R+0JZr7DPj19XwcgFnyMIgU181OvkkifRwTK+5XXf2IbqPp14qXfUnLq5aQq1x8vm/OAcjYnSQjVJ9oO39yBTh9hcy1qXJ3qGneWOfF0e6m9wPtgDp/zuujOM9uF3OPxw/95fKdCG5zYQ= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: d5ed07ae-ff2d-492e-29b8-08d5172220d7 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254172)(48565401081)(4534020)(4602075)(4627075)(201703031133081)(201702281549075)(2017052603199)(201703131423095); SRVR:DB6PR0801MB2053; x-ms-traffictypediagnostic: DB6PR0801MB2053: nodisclaimer: True x-exchange-antispam-report-test: UriScan:(180628864354917); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(10201501046)(100000703101)(100105400095)(3002001)(93006095)(93001095)(6055026)(6041248)(20161123564025)(20161123558100)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DB6PR0801MB2053; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DB6PR0801MB2053; x-forefront-prvs: 0465429B7F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(376002)(39860400002)(346002)(377424004)(24454002)(189002)(54534003)(199003)(105586002)(106356001)(8676002)(8936002)(229853002)(68736007)(575784001)(2906002)(86362001)(81156014)(54356999)(101416001)(81166006)(76176999)(33656002)(6436002)(50986999)(3660700001)(3280700002)(97736004)(4001150100001)(6506006)(189998001)(6916009)(5250100002)(478600001)(5660300001)(7696004)(2950100002)(72206003)(3846002)(102836003)(6116002)(53936002)(6246003)(55016002)(2900100001)(99286003)(74316002)(316002)(25786009)(54906003)(305945005)(66066001)(9686003)(7736002)(14454004)(4326008); DIR:OUT; SFP:1101; SCL:1; SRVR:DB6PR0801MB2053; H:DB6PR0801MB2053.eurprd08.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:3; A:1; LANG:en; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Oct 2017 18:49:28.9623 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0801MB2053 DJ Delorie wrote: > > +    if (SINGLE_THREAD_P) >>        { > > If you set have_lock to zero here, you can omit the last two chunks of > this patch. Here is the updated version with that change and the original check: This patch adds single-threaded fast paths to _int_free. Bypass the explicit locking for larger allocations. Passes GLIBC tests, OK for commit? ChangeLog: 2017-10-19 Wilco Dijkstra * malloc/malloc.c (_int_free): Add SINGLE_THREAD_P paths. diff --git a/malloc/malloc.c b/malloc/malloc.c index e220fba83b0f9dc515aef562bdfca6a3ad13d3ea..ca5cfff3a1b1882ae608219fdec973b7f13cbb21 100644 --- a/malloc/malloc.c +++ b/malloc/malloc.c @@ -4195,24 +4195,34 @@ _int_free (mstate av, mchunkptr p, int have_lock) /* Atomically link P to its fastbin: P->FD = *FB; *FB = P; */ mchunkptr old = *fb, old2; - unsigned int old_idx = ~0u; - do + + if (SINGLE_THREAD_P) { - /* Check that the top of the bin is not the record we are going to add - (i.e., double free). */ + /* Check that the top of the bin is not the record we are going to + add (i.e., double free). */ if (__builtin_expect (old == p, 0)) malloc_printerr ("double free or corruption (fasttop)"); - /* Check that size of fastbin chunk at the top is the same as - size of the chunk that we are adding. We can dereference OLD - only if we have the lock, otherwise it might have already been - deallocated. See use of OLD_IDX below for the actual check. */ - if (have_lock && old != NULL) - old_idx = fastbin_index(chunksize(old)); - p->fd = old2 = old; + p->fd = old; + *fb = p; } - while ((old = catomic_compare_and_exchange_val_rel (fb, p, old2)) != old2); - - if (have_lock && old != NULL && __builtin_expect (old_idx != idx, 0)) + else + do + { + /* Check that the top of the bin is not the record we are going to + add (i.e., double free). */ + if (__builtin_expect (old == p, 0)) + malloc_printerr ("double free or corruption (fasttop)"); + p->fd = old2 = old; + } + while ((old = catomic_compare_and_exchange_val_rel (fb, p, old2)) + != old2); + + /* Check that size of fastbin chunk at the top is the same as + size of the chunk that we are adding. We can dereference OLD + only if we have the lock, otherwise it might have already been + allocated again. */ + if (have_lock && old != NULL + && __builtin_expect (fastbin_index (chunksize (old)) != idx, 0)) malloc_printerr ("invalid fastbin entry (free)"); } @@ -4221,6 +4231,11 @@ _int_free (mstate av, mchunkptr p, int have_lock) */ else if (!chunk_is_mmapped(p)) { + + /* If we're single-threaded, don't lock the arena. */ + if (SINGLE_THREAD_P) + have_lock = true; + if (!have_lock) __libc_lock_lock (av->mutex);