From patchwork Fri Jun 11 04:08:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liudongyun X-Patchwork-Id: 43830 X-Patchwork-Delegate: siddhesh@gotplt.org Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id E81C13939C02 for ; Fri, 11 Jun 2021 04:10:36 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from h3cspam02-ex.h3c.com (smtp.h3c.com [60.191.123.50]) by sourceware.org (Postfix) with ESMTPS id 36F7F385700A; Fri, 11 Jun 2021 04:10:22 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 36F7F385700A Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=h3c.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=h3c.com Received: from DAG2EX03-BASE.srv.huawei-3com.com ([10.8.0.66]) by h3cspam02-ex.h3c.com with ESMTP id 15B49L7B065790; Fri, 11 Jun 2021 12:09:21 +0800 (GMT-8) (envelope-from liu.dongyun@h3c.com) Received: from localhost.localdomain (10.114.244.30) by DAG2EX03-BASE.srv.huawei-3com.com (10.8.0.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4; Fri, 11 Jun 2021 12:09:22 +0800 From: liudongyun To: Subject: [PATCH] release all idle heaps in the non-main-arena Date: Fri, 11 Jun 2021 12:08:52 +0800 Message-ID: <20210611040852.21745-1-liu.dongyun@h3c.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-Originating-IP: [10.114.244.30] X-ClientProxiedBy: BJSMTP01-EX.srv.huawei-3com.com (10.63.20.132) To DAG2EX03-BASE.srv.huawei-3com.com (10.8.0.66) X-DNSRBL: X-MAIL: h3cspam02-ex.h3c.com 15B49L7B065790 X-Spam-Status: No, score=-13.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libc-locales@sourceware.org, libc-maintainers@gnu.org, deng.hongjie@h3c.com, liu.dongyun@h3c.com, dai.xianjun@h3c.com Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" liu.dongyun@h3c.com report this question in follow: The system has a total of 32G memory and 12 core cpu. After frequent malloc and free memory by multiple threads, we found that many physical memory that should have been released have not been released.We observe that the maximum free memory can reach 15G. deng.hongjie@h3c.com recommend reducing the heap size of the non-main-arena, but it doesn't work. liu.dongyun@h3c.com found When the last heap in the user process into the non-main-area has memory in use, the previous heap will not be released, even if the previous heap is already free. This mechanism resulting in the memory cache reaching 15G. liu.dongyun@h3c.com tried to solve this problem by checking to release the heap that the current chunk belongs to and unmmaped if it is all free. Then we adjust the heap size of the non-main-arena to 512KB. The method has achieved good results. Submit patch only to modify one of them. The test after modification has the following results. With business: -------------------------------------------------------------------------------- Memory statistics are measured in KB: Slot 0: Total Used Free Shared Buffers Cached FreeRatio Mem: 32598612 7501484 25097128 0 1812 2121380 77.0% -------------------------------------------------------------------------------- A few minutes after deleted business: Memory statistics are measured in KB: Slot 0: Total Used Free Shared Buffers Cached FreeRatio Mem: 32598612 9763528 22835084 0 32 2118208 70.0% -------------------------------------------------------------------------------- Bugzilla: https://sourceware.org/bugzilla/show_bug.cgi?id=27976 Reported-by: liudongyun and Reported-by: denghongjie --- malloc/arena.c | 30 +++++++++++++++++++++++++++++- malloc/malloc.c | 2 +- 2 files changed, 30 insertions(+), 2 deletions(-) mode change 100644 => 100755 malloc/arena.c mode change 100644 => 100755 malloc/malloc.c diff --git a/malloc/arena.c b/malloc/arena.c old mode 100644 new mode 100755 index 7eb110445e..7812997886 --- a/malloc/arena.c +++ b/malloc/arena.c @@ -606,14 +606,42 @@ shrink_heap (heap_info *h, long diff) } while (0) static int -heap_trim (heap_info *heap, size_t pad) +heap_trim(heap_info *heap, heap_info *free_heap, size_t pad) { mstate ar_ptr = heap->ar_ptr; unsigned long pagesz = GLRO (dl_pagesize); mchunkptr top_chunk = top (ar_ptr), p; heap_info *prev_heap; long new_size, top_size, top_area, extra, prev_size, misalign; + heap_info *last_heap; + /*release part of the cache memory*/ + last_heap = heap_for_ptr(top_chunk); + if ((NULL != free_heap->prev) && (last_heap != free_heap)) + { + p = chunk_at_offset(free_heap, sizeof(*heap)); + if (!inuse(p)) + { + if (chunksize(p) + sizeof(*free_heap) + MINSIZE == free_heap->size) + { + while(last_heap) + { + if (last_heap->prev == free_heap) + { + last_heap->prev = free_heap->prev; + break; + } + last_heap = last_heap->prev; + } + ar_ptr->system_mem -= free_heap->size; + arena_mem -= free_heap->size; + unlink_chunk(ar_ptr, p); + delete_heap(free_heap); + return 1; + } + } + } + /* Can this heap go away completely? */ while (top_chunk == chunk_at_offset (heap, sizeof (*heap))) { diff --git a/malloc/malloc.c b/malloc/malloc.c old mode 100644 new mode 100755 index 0e2e1747e0..40f0a8e458 --- a/malloc/malloc.c +++ b/malloc/malloc.c @@ -4665,7 +4665,7 @@ _int_free (mstate av, mchunkptr p, int have_lock) heap_info *heap = heap_for_ptr(top(av)); assert(heap->ar_ptr == av); - heap_trim(heap, mp_.top_pad); + heap_trim(heap, heap_for_ptr(p), mp_.top_pad); } }