From patchwork Sun Mar 17 12:19:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 87286 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 170FC38582A2 for ; Sun, 17 Mar 2024 12:20:23 +0000 (GMT) X-Original-To: binutils@sourceware.org Delivered-To: binutils@sourceware.org Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by sourceware.org (Postfix) with ESMTPS id 4B6023858CD1 for ; Sun, 17 Mar 2024 12:19:17 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4B6023858CD1 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 4B6023858CD1 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::42a ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677961; cv=none; b=PxZzz3bCsLRvjVdbbG6yx76MBYACgv2ychn1kwe3PzNT5jJeMEjLHFTbz7JDQ0EskTkM8TzahfPjxfCtm+LsAyViMdjRmx9y/NHmppwYxbk0ehthEgaFNkWPazwsvKbuCHeU8/dwxiiMHDrwKrrpySMwylG1SjoKicqRU3G+auo= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677961; c=relaxed/simple; bh=+jnyMc2f4yIw+blArK6ERGeAdg+Sd2cJ9Mhe0FnRnEw=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=Jj1+M/5EPifgVr5HEDLNf/SGY5LwwZsAWhwye6EOkbVo1pgmkKhaA99z4BkqIfQigJphzUJDaFVLTS8daadRRCHxLNY91NZfEK1Omrl6lwgA5EWH5OnKWjMhLh8RJmc2UpRcgkdbS926sAGopJlIimxo1BA/k8PbUEQM4bafjNU= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-6e6c0098328so2923427b3a.3 for ; Sun, 17 Mar 2024 05:19:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710677956; x=1711282756; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=orBT0KACp4P6NJcM0hGDO37dKoUKitUWAjDw+8fd8Zs=; b=d5XOciMo/ezxNMEESrQCRDFOm+mhqYDYNIjvHm07SG5mZn1aRg4KJIOXI120h33MFQ IRdzueBJ9dj0qfbytSK7mCECWyBOX9Y1jtNMCzjiCpV9FnO5CMIYMkZF5HsveOvo4QEM FJwGZj/kBW2yQ703s3hBOhiKbiWKqrNeDtAKBAy+619Y82l8/ZuKbwAx78Efdvxwz1Fg 7s8JWt2jwSF511uFRmjqTqs+3mBp+LRpFTkrakNUJlQMGkF9TifuRG4rk7QOChj2zhYS 8/Fqb9ZMyVBuXHI/Z7c/9vz5eIsd/qGkOsX3gippuTj/NcC/njxsntrpuU+F8gv4kMxL 7JvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710677956; x=1711282756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=orBT0KACp4P6NJcM0hGDO37dKoUKitUWAjDw+8fd8Zs=; b=j8j9s8bAUvTdum+yLdNyS4fYXlz2mTPAc32mqss4hQyXcSc2IC/P2cKtmFQL/1WoEF uqodC1VB88JoiyXV/BJ9If1o8dA9nKcwwpRoeKGrBWRpoQJptVpap5F04RWakVr7hTNk 7zdHo7dLmCkRS2clJ5SCh62ANce/ToskpOVmp3BlDqfSS03H9YmAms7qT8E8T/CtQIzS BbqGK92siMNcf3ckHmx5TANC+RmeqY2wRGmhzSMed/CDoU2DQbQy2NF0pHLX1bd38hbO 73c6iLEkU2p3x3fgTFlM37RjCn6afvsF/Y6ExqH/6IQJKe55Vf0Wzza6Nf/0lROiIuzp CqjA== X-Gm-Message-State: AOJu0YyLxPXoBSnzO/2fa0/GwqwRoAbWopxftfr3IFrCuqiXfVOq6ias SUTs3FFAY1uQzd7Olz0ufKLGhuFLbxLMCRHUEIjyWSLeFJa/YSYM/sPvEkXv X-Google-Smtp-Source: AGHT+IGvkhYYJGMhcwMb8LD/Gl40MAr6xs+TbeDrTHXxF7x7S/LtYco/j6ekk0iN8skFWTN8MSTlUA== X-Received: by 2002:a05:6a00:9388:b0:6e6:8ca5:c0d4 with SMTP id ka8-20020a056a00938800b006e68ca5c0d4mr11000887pfb.12.1710677955988; Sun, 17 Mar 2024 05:19:15 -0700 (PDT) Received: from gnu-cfl-3.localdomain ([172.58.89.72]) by smtp.gmail.com with ESMTPSA id u14-20020aa7848e000000b006e6b4c637b6sm6127629pfn.116.2024.03.17.05.19.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 17 Mar 2024 05:19:14 -0700 (PDT) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id DB38D740337; Sun, 17 Mar 2024 05:19:12 -0700 (PDT) From: "H.J. Lu" To: binutils@sourceware.org Cc: goldstein.w.n@gmail.com, sam@gentoo.org, amodra@gmail.com Subject: [PATCH v12 1/6] elf: Use mmap to map in read-only sections Date: Sun, 17 Mar 2024 05:19:07 -0700 Message-ID: <20240317121912.799372-2-hjl.tools@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240317121912.799372-1-hjl.tools@gmail.com> References: <20240317121912.799372-1-hjl.tools@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-3019.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_ABUSEAT, RCVD_IN_DNSWL_NONE, RCVD_IN_SBL_CSS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: binutils-bounces+patchwork=sourceware.org@sourceware.org There are many linker input files in LLVM debug build with huge string sections. All these string sections can be treated as read-only. But linker copies all of them into memory which consumes huge amount of memory and slows down linker significantly. Add _bfd_mmap_readonly_persistent and _bfd_mmap_readonly_temporary to mmap in reado-only sections with size >= 4 * page size. NB: All string sections in valid ELF inputs must be null terminated. There is no need to terminate it again and string sections are mmapped as read-only. * bfd.c (bfd_mmapped_entry): New. (bfd_mmapped): Likewise. (bfd): Add mmapped. * bfdwin.c (bfd_get_file_window): Use _bfd_pagesize. * cache.c (cache_bmmap): Remove pagesize_m1 and use pagesize_m1 instead. * elf.c (bfd_elf_get_str_section): Call _bfd_mmap_readonly_persistent instead of _bfd_alloc_and_read. Don't terminate the string section again. (get_hash_table_data): Call _bfd_mmap_readonly_temporary and _bfd_munmap_readonly_temporary instead of _bfd_malloc_and_read and free. (_bfd_elf_get_dynamic_symbols): Call _bfd_mmap_readonly_persistent instead of _bfd_alloc_and_read. Don't terminate the string section again. Call _bfd_mmap_readonly_temporary and _bfd_munmap_readonly_temporary instead of _bfd_malloc_and_read and free. (_bfd_elf_slurp_version_tables): Call _bfd_mmap_readonly_temporary and _bfd_munmap_readonly_temporary instead of _bfd_malloc_and_read and free. * elflink.c (bfd_elf_link_record_dynamic_symbol): Use bfd_malloc to get the unversioned symbol. * libbfd-in.h (_bfd_pagesize): New. (_bfd_pagesize_m1): Likewise. (_bfd_minimum_mmap_size): Likewise. (_bfd_mmap_readonly_persistent): Likewise. (_bfd_mmap_readonly_temporary): Likewise. (_bfd_munmap_readonly_temporary): Likewise. * libbfd.c (bfd_allocate_mmapped_page): New. (_bfd_mmap_readonly_temporary): Likewise. (_bfd_munmap_readonly_temporary): Likewise. (_bfd_mmap_readonly_persistent): Likewise. (_bfd_pagesize): Likewise. (_bfd_pagesize_m1): Likewise. (_bfd_minimum_mmap_size): Likewise. (bfd_init_pagesize): Likewise. * lynx-core.c (lynx_core_file_p): Use _bfd_pagesize. * opncls.c (_bfd_delete_bfd): Munmap tracked mmapped memories. * sysdep.h (MAP_ANONYMOUS): New. Define if undefined. * bfd-in2.h: Regenerated. * libbfd.h: Likewise. --- bfd/bfd-in2.h | 17 ++++++ bfd/bfd.c | 17 ++++++ bfd/bfdwin.c | 8 +-- bfd/cache.c | 7 +-- bfd/elf.c | 79 +++++++++++++++++-------- bfd/elflink.c | 16 ++--- bfd/libbfd-in.h | 29 ++++++++++ bfd/libbfd.c | 151 ++++++++++++++++++++++++++++++++++++++++++++++++ bfd/libbfd.h | 29 ++++++++++ bfd/lynx-core.c | 2 +- bfd/opncls.c | 12 ++++ bfd/sysdep.h | 4 ++ 12 files changed, 328 insertions(+), 43 deletions(-) diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h index 29602e054da..544422a9522 100644 --- a/bfd/bfd-in2.h +++ b/bfd/bfd-in2.h @@ -1951,6 +1951,20 @@ struct bfd_build_id bfd_byte data[1]; }; +struct bfd_mmapped_entry + { + void *addr; + size_t size; + }; + +struct bfd_mmapped + { + struct bfd_mmapped *next; + unsigned int max_entry; + unsigned int next_entry; + struct bfd_mmapped_entry entries[1]; + }; + struct bfd { /* The filename the application opened the BFD with. */ @@ -2280,6 +2294,9 @@ struct bfd /* For input BFDs, the build ID, if the object has one. */ const struct bfd_build_id *build_id; + + /* For input BFDs, mmapped entries. */ + struct bfd_mmapped *mmapped; }; static inline const char * diff --git a/bfd/bfd.c b/bfd/bfd.c index 54061a34240..8302013f1b4 100644 --- a/bfd/bfd.c +++ b/bfd/bfd.c @@ -74,6 +74,20 @@ EXTERNAL . bfd_byte data[1]; . }; . +.struct bfd_mmapped_entry +. { +. void *addr; +. size_t size; +. }; +. +.struct bfd_mmapped +. { +. struct bfd_mmapped *next; +. unsigned int max_entry; +. unsigned int next_entry; +. struct bfd_mmapped_entry entries[1]; +. }; +. CODE_FRAGMENT .struct bfd @@ -406,6 +420,9 @@ CODE_FRAGMENT . . {* For input BFDs, the build ID, if the object has one. *} . const struct bfd_build_id *build_id; +. +. {* For input BFDs, mmapped entries. *} +. struct bfd_mmapped *mmapped; .}; . diff --git a/bfd/bfdwin.c b/bfd/bfdwin.c index 2919c71b3cb..73e44635bcb 100644 --- a/bfd/bfdwin.c +++ b/bfd/bfdwin.c @@ -157,7 +157,7 @@ bfd_get_file_window (bfd *abfd, bool writable) { static int ok_to_map = 1; - static size_t pagesize; + size_t pagesize = _bfd_pagesize; bfd_window_internal *i = windowp->i; bfd_size_type size_to_alloc = size; @@ -167,12 +167,6 @@ bfd_get_file_window (bfd *abfd, windowp, windowp->data, (unsigned long) windowp->size, windowp->i, writable); - /* Make sure we know the page size, so we can be friendly to mmap. */ - if (pagesize == 0) - pagesize = getpagesize (); - if (pagesize == 0) - abort (); - if (i == NULL) { i = bfd_zmalloc (sizeof (bfd_window_internal)); diff --git a/bfd/cache.c b/bfd/cache.c index d0e7be293a5..0f994c74239 100644 --- a/bfd/cache.c +++ b/bfd/cache.c @@ -494,10 +494,10 @@ cache_bmmap (struct bfd *abfd ATTRIBUTE_UNUSED, #ifdef HAVE_MMAP else { - static uintptr_t pagesize_m1; + uintptr_t pagesize_m1 = _bfd_pagesize_m1; FILE *f; file_ptr pg_offset; - bfd_size_type pg_len; + size_t pg_len; f = bfd_cache_lookup (abfd, CACHE_NO_SEEK_ERROR); if (f == NULL) @@ -506,9 +506,6 @@ cache_bmmap (struct bfd *abfd ATTRIBUTE_UNUSED, return ret; } - if (pagesize_m1 == 0) - pagesize_m1 = getpagesize () - 1; - /* Align. */ pg_offset = offset & ~pagesize_m1; pg_len = (len + (offset - pg_offset) + pagesize_m1) & ~pagesize_m1; diff --git a/bfd/elf.c b/bfd/elf.c index 8bffd3c5141..c80fff47b45 100644 --- a/bfd/elf.c +++ b/bfd/elf.c @@ -289,16 +289,23 @@ bfd_elf_get_str_section (bfd *abfd, unsigned int shindex) in case the string table is not terminated. */ if (shstrtabsize + 1 <= 1 || bfd_seek (abfd, offset, SEEK_SET) != 0 - || (shstrtab = _bfd_alloc_and_read (abfd, shstrtabsize + 1, - shstrtabsize)) == NULL) + || (shstrtab + = _bfd_mmap_readonly_persistent (abfd, shstrtabsize)) == NULL) { /* Once we've failed to read it, make sure we don't keep trying. Otherwise, we'll keep allocating space for the string table over and over. */ i_shdrp[shindex]->sh_size = 0; } - else - shstrtab[shstrtabsize] = '\0'; + else if (shstrtab[shstrtabsize - 1] != '\0') + { + /* It is an error if a string table isn't terminated. */ + _bfd_error_handler + /* xgettext:c-format */ + (_("%pB(%pA): string table is corrupt"), + abfd, i_shdrp[shindex]->bfd_section); + return NULL; + } i_shdrp[shindex]->contents = shstrtab; } return (char *) shstrtab; @@ -1919,6 +1926,8 @@ get_hash_table_data (bfd *abfd, bfd_size_type number, unsigned char *e_data = NULL; bfd_vma *i_data = NULL; bfd_size_type size; + void *e_data_addr; + size_t e_data_size ATTRIBUTE_UNUSED; if (ent_size != 4 && ent_size != 8) return NULL; @@ -1940,7 +1949,8 @@ get_hash_table_data (bfd *abfd, bfd_size_type number, return NULL; } - e_data = _bfd_malloc_and_read (abfd, size, size); + e_data = _bfd_mmap_readonly_temporary (abfd, size, &e_data_addr, + &e_data_size); if (e_data == NULL) return NULL; @@ -1958,7 +1968,7 @@ get_hash_table_data (bfd *abfd, bfd_size_type number, while (number--) i_data[number] = bfd_get_64 (abfd, e_data + number * ent_size); - free (e_data); + _bfd_munmap_readonly_temporary (e_data_addr, e_data_size); return i_data; } @@ -2007,6 +2017,10 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, size_t verneed_size = 0; size_t extsym_size; const struct elf_backend_data *bed; + void *dynbuf_addr = NULL; + void *esymbuf_addr = NULL; + size_t dynbuf_size = 0; + size_t esymbuf_size = 0; /* Return TRUE if symbol table is bad. */ if (elf_bad_symtab (abfd)) @@ -2024,7 +2038,9 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, if (bfd_seek (abfd, phdr->p_offset, SEEK_SET) != 0) goto error_return; - dynbuf = _bfd_malloc_and_read (abfd, phdr->p_filesz, phdr->p_filesz); + dynbuf_size = phdr->p_filesz; + dynbuf = _bfd_mmap_readonly_temporary (abfd, dynbuf_size, + &dynbuf_addr, &dynbuf_size); if (dynbuf == NULL) goto error_return; @@ -2102,11 +2118,17 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, goto error_return; /* Dynamic string table must be valid until ABFD is closed. */ - strbuf = (char *) _bfd_alloc_and_read (abfd, dt_strsz + 1, dt_strsz); + strbuf = (char *) _bfd_mmap_readonly_persistent (abfd, dt_strsz); if (strbuf == NULL) goto error_return; - /* Since this is a string table, make sure that it is terminated. */ - strbuf[dt_strsz] = 0; + if (strbuf[dt_strsz - 1] != 0) + { + /* It is an error if a string table is't terminated. */ + _bfd_error_handler + /* xgettext:c-format */ + (_("%pB: DT_STRTAB table is corrupt"), abfd); + goto error_return; + } /* Get the real symbol count from DT_HASH or DT_GNU_HASH. Prefer DT_HASH since it is simpler than DT_GNU_HASH. */ @@ -2281,7 +2303,10 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, if (filepos == (file_ptr) -1 || bfd_seek (abfd, filepos, SEEK_SET) != 0) goto error_return; - esymbuf = _bfd_malloc_and_read (abfd, amt, amt); + esymbuf_size = amt; + esymbuf = _bfd_mmap_readonly_temporary (abfd, esymbuf_size, + &esymbuf_addr, + &esymbuf_size); if (esymbuf == NULL) goto error_return; @@ -2325,7 +2350,7 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, goto error_return; /* DT_VERSYM info must be valid until ABFD is closed. */ - versym = _bfd_alloc_and_read (abfd, amt, amt); + versym = _bfd_mmap_readonly_persistent (abfd, amt); if (dt_verdef) { @@ -2337,8 +2362,7 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, goto error_return; /* DT_VERDEF info must be valid until ABFD is closed. */ - verdef = _bfd_alloc_and_read (abfd, verdef_size, - verdef_size); + verdef = _bfd_mmap_readonly_persistent (abfd, verdef_size); } if (dt_verneed) @@ -2351,8 +2375,7 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, goto error_return; /* DT_VERNEED info must be valid until ABFD is closed. */ - verneed = _bfd_alloc_and_read (abfd, verneed_size, - verneed_size); + verneed = _bfd_mmap_readonly_persistent (abfd, verneed_size); } } @@ -2375,8 +2398,8 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, /* Restore file position for elf_object_p. */ if (bfd_seek (abfd, saved_filepos, SEEK_SET) != 0) res = false; - free (dynbuf); - free (esymbuf); + _bfd_munmap_readonly_temporary (dynbuf_addr, dynbuf_size); + _bfd_munmap_readonly_temporary (esymbuf_addr, esymbuf_size); free (gnubuckets); free (gnuchains); free (mipsxlat); @@ -9435,6 +9458,8 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) bfd_byte *contents = NULL; unsigned int freeidx = 0; size_t amt; + void *contents_addr = NULL; + size_t contents_size = 0; if (elf_dynverref (abfd) != 0 || elf_tdata (abfd)->dt_verneed != NULL) { @@ -9471,7 +9496,10 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) if (bfd_seek (abfd, hdr->sh_offset, SEEK_SET) != 0) goto error_return_verref; - contents = _bfd_malloc_and_read (abfd, hdr->sh_size, hdr->sh_size); + contents_size = hdr->sh_size; + contents = _bfd_mmap_readonly_temporary (abfd, contents_size, + &contents_addr, + &contents_size); if (contents == NULL) goto error_return_verref; @@ -9604,8 +9632,9 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) elf_tdata (abfd)->cverrefs = i; if (contents != elf_tdata (abfd)->dt_verneed) - free (contents); + _bfd_munmap_readonly_temporary (contents_addr, contents_size); contents = NULL; + contents_addr = NULL; } if (elf_dynverdef (abfd) != 0 || elf_tdata (abfd)->dt_verdef != NULL) @@ -9646,7 +9675,10 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) if (bfd_seek (abfd, hdr->sh_offset, SEEK_SET) != 0) goto error_return_verdef; - contents = _bfd_malloc_and_read (abfd, hdr->sh_size, hdr->sh_size); + contents_size = hdr->sh_size; + contents = _bfd_mmap_readonly_temporary (abfd, contents_size, + &contents_addr, + &contents_size); if (contents == NULL) goto error_return_verdef; @@ -9800,8 +9832,9 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) } if (contents != elf_tdata (abfd)->dt_verdef) - free (contents); + _bfd_munmap_readonly_temporary (contents_addr, contents_size); contents = NULL; + contents_addr = NULL; } else if (default_imported_symver) { @@ -9857,7 +9890,7 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) error_return: if (contents != elf_tdata (abfd)->dt_verneed && contents != elf_tdata (abfd)->dt_verdef) - free (contents); + _bfd_munmap_readonly_temporary (contents_addr, contents_size); return false; } diff --git a/bfd/elflink.c b/bfd/elflink.c index 5a6cb07b2ce..42029f29f7a 100644 --- a/bfd/elflink.c +++ b/bfd/elflink.c @@ -549,22 +549,24 @@ bfd_elf_link_record_dynamic_symbol (struct bfd_link_info *info, return false; } + char *unversioned_name = NULL; + /* We don't put any version information in the dynamic string table. */ name = h->root.root.string; p = strchr (name, ELF_VER_CHR); if (p != NULL) - /* We know that the p points into writable memory. In fact, - there are only a few symbols that have read-only names, being - those like _GLOBAL_OFFSET_TABLE_ that are created specially - by the backends. Most symbols will have names pointing into - an ELF string table read from a file, or to objalloc memory. */ - *p = 0; + { + unversioned_name = bfd_malloc (p - name + 1); + memcpy (unversioned_name, name, p - name); + unversioned_name[p - name] = 0; + name = unversioned_name; + } indx = _bfd_elf_strtab_add (dynstr, name, p != NULL); if (p != NULL) - *p = ELF_VER_CHR; + free (unversioned_name); if (indx == (size_t) -1) return false; diff --git a/bfd/libbfd-in.h b/bfd/libbfd-in.h index b8b2ce7ba09..c5a79cf932c 100644 --- a/bfd/libbfd-in.h +++ b/bfd/libbfd-in.h @@ -851,6 +851,10 @@ extern struct bfd_link_info *_bfd_get_link_info (bfd *) extern bool _bfd_link_keep_memory (struct bfd_link_info *) ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_pagesize ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_pagesize_m1 ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_minimum_mmap_size ATTRIBUTE_HIDDEN; + #if GCC_VERSION >= 7000 #define _bfd_mul_overflow(a, b, res) __builtin_mul_overflow (a, b, res) #else @@ -888,6 +892,19 @@ _bfd_alloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) return NULL; } +#ifdef USE_MMAP +extern void *_bfd_mmap_readonly_persistent + (bfd *, size_t) ATTRIBUTE_HIDDEN; +extern void *_bfd_mmap_readonly_temporary + (bfd *, size_t, void **, size_t *) ATTRIBUTE_HIDDEN; +extern void _bfd_munmap_readonly_temporary + (void *, size_t) ATTRIBUTE_HIDDEN; +#else +#define _bfd_mmap_readonly_persistent(abfd, rsize) \ + _bfd_alloc_and_read (abfd, rsize, rsize) +#define _bfd_munmap_readonly_temporary(ptr, rsize) free (ptr) +#endif + static inline void * _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) { @@ -910,3 +927,15 @@ _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) } return NULL; } + +#ifndef USE_MMAP +static inline void * +_bfd_mmap_readonly_temporary (bfd *abfd, size_t rsize, void **map_addr, + size_t *map_size) +{ + void *mem = _bfd_malloc_and_read (abfd, rsize, rsize); + *map_addr = mem; + *map_size = rsize; + return mem; +} +#endif diff --git a/bfd/libbfd.c b/bfd/libbfd.c index f8d148c9677..a79c814a0dc 100644 --- a/bfd/libbfd.c +++ b/bfd/libbfd.c @@ -1038,6 +1038,141 @@ bfd_get_bits (const void *p, int bits, bool big_p) return data; } +#ifdef USE_MMAP +/* Allocate a page to track mmapped memory and return the page and + the first entry. Return NULL if mmap fails. */ + +static struct bfd_mmapped * +bfd_allocate_mmapped_page (bfd *abfd, struct bfd_mmapped_entry **entry) +{ + struct bfd_mmapped * mmapped + = (struct bfd_mmapped *) mmap (NULL, _bfd_pagesize, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, + -1, 0); + if (mmapped == MAP_FAILED) + return NULL; + + mmapped->next = abfd->mmapped; + mmapped->max_entry + = ((_bfd_pagesize - offsetof (struct bfd_mmapped, entries)) + / sizeof (struct bfd_mmapped_entry)); + mmapped->next_entry = 1; + abfd->mmapped = mmapped; + *entry = mmapped->entries; + return mmapped; +} + +/* Mmap a memory region of RSIZE bytes with PROT at the current offset. + Return mmap address and size in MAP_ADDR and MAP_SIZE. Return NULL + on invalid input and MAP_FAILED for mmap failure. */ + +static void * +bfd_mmap_local (bfd *abfd, size_t rsize, int prot, void **map_addr, + size_t *map_size) +{ + if (!_bfd_constant_p (rsize)) + { + ufile_ptr filesize = bfd_get_file_size (abfd); + if (filesize != 0 && rsize > filesize) + { + bfd_set_error (bfd_error_file_truncated); + return NULL; + } + } + + void *mem; + ufile_ptr offset = bfd_tell (abfd); + mem = bfd_mmap (abfd, NULL, rsize, prot, MAP_PRIVATE, offset, + map_addr, map_size); + return mem; +} + +/* Mmap a readonly memory region of RSIZE bytes at the current offset. + Return mmap address and size in MAP_ADDR and MAP_SIZE. Return NULL + on invalid input and MAP_FAILED for mmap failure. */ + +void * +_bfd_mmap_readonly_temporary (bfd *abfd, size_t rsize, void **map_addr, + size_t *map_size) +{ + /* Use mmap only if section size >= the minimum mmap section size. */ + if (rsize < _bfd_minimum_mmap_size) + { + void *mem = _bfd_malloc_and_read (abfd, rsize, rsize); + /* NB: Set *MAP_ADDR to MEM and *MAP_SIZE to 0 to indicate that + _bfd_malloc_and_read is called. */ + *map_addr = mem; + *map_size = 0; + return mem; + } + + return bfd_mmap_local (abfd, rsize, PROT_READ, map_addr, map_size); +} + +/* Munmap RSIZE bytes at PTR. */ + +void +_bfd_munmap_readonly_temporary (void *ptr, size_t rsize) +{ + /* NB: Since _bfd_munmap_readonly_temporary is called like free, PTR + may be NULL. Otherwise, PTR and RSIZE must be valid. If RSIZE is + 0, _bfd_malloc_and_read is called. */ + if (ptr == NULL) + return; + if (rsize != 0) + { + if (munmap (ptr, rsize) != 0) + abort (); + } + else + free (ptr); +} + +/* Mmap a readonly memory region of RSIZE bytes at the current offset. + Return NULL on invalid input or mmap failure. */ + +void * +_bfd_mmap_readonly_persistent (bfd *abfd, size_t rsize) +{ + /* Use mmap only if section size >= the minimum mmap section size. */ + if (rsize < _bfd_minimum_mmap_size) + return _bfd_alloc_and_read (abfd, rsize, rsize); + + void *mem, *map_addr; + size_t map_size; + mem = bfd_mmap_local (abfd, rsize, PROT_READ, &map_addr, &map_size); + if (mem == NULL) + return mem; + if (mem == MAP_FAILED) + return _bfd_alloc_and_read (abfd, rsize, rsize); + + struct bfd_mmapped_entry *entry; + unsigned int next_entry; + struct bfd_mmapped *mmapped = abfd->mmapped; + if (mmapped != NULL + && (next_entry = mmapped->next_entry) < mmapped->max_entry) + { + entry = &mmapped->entries[next_entry]; + mmapped->next_entry++; + } + else + { + mmapped = bfd_allocate_mmapped_page (abfd, &entry); + if (mmapped == NULL) + { + munmap (map_addr, map_size); + return NULL; + } + } + + entry->addr = map_addr; + entry->size = map_size; + + return mem; +} +#endif + /* Default implementation */ bool @@ -1326,3 +1461,19 @@ _bfd_generic_init_private_section_data (bfd *ibfd ATTRIBUTE_UNUSED, { return true; } + +uintptr_t _bfd_pagesize; +uintptr_t _bfd_pagesize_m1; +uintptr_t _bfd_minimum_mmap_size; + +__attribute__ ((unused, constructor)) +static void +bfd_init_pagesize (void) +{ + _bfd_pagesize = getpagesize (); + if (_bfd_pagesize == 0) + abort (); + _bfd_pagesize_m1 = _bfd_pagesize - 1; + /* The minimum section size to use mmap. */ + _bfd_minimum_mmap_size = _bfd_pagesize * 4; +} diff --git a/bfd/libbfd.h b/bfd/libbfd.h index f15b5f27db8..47f40889a95 100644 --- a/bfd/libbfd.h +++ b/bfd/libbfd.h @@ -857,6 +857,10 @@ extern struct bfd_link_info *_bfd_get_link_info (bfd *) extern bool _bfd_link_keep_memory (struct bfd_link_info *) ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_pagesize ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_pagesize_m1 ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_minimum_mmap_size ATTRIBUTE_HIDDEN; + #if GCC_VERSION >= 7000 #define _bfd_mul_overflow(a, b, res) __builtin_mul_overflow (a, b, res) #else @@ -894,6 +898,19 @@ _bfd_alloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) return NULL; } +#ifdef USE_MMAP +extern void *_bfd_mmap_readonly_persistent + (bfd *, size_t) ATTRIBUTE_HIDDEN; +extern void *_bfd_mmap_readonly_temporary + (bfd *, size_t, void **, size_t *) ATTRIBUTE_HIDDEN; +extern void _bfd_munmap_readonly_temporary + (void *, size_t) ATTRIBUTE_HIDDEN; +#else +#define _bfd_mmap_readonly_persistent(abfd, rsize) \ + _bfd_alloc_and_read (abfd, rsize, rsize) +#define _bfd_munmap_readonly_temporary(ptr, rsize) free (ptr) +#endif + static inline void * _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) { @@ -916,6 +933,18 @@ _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) } return NULL; } + +#ifndef USE_MMAP +static inline void * +_bfd_mmap_readonly_temporary (bfd *abfd, size_t rsize, void **map_addr, + size_t *map_size) +{ + void *mem = _bfd_malloc_and_read (abfd, rsize, rsize); + *map_addr = mem; + *map_size = rsize; + return mem; +} +#endif /* Extracted from libbfd.c. */ void *bfd_malloc (bfd_size_type /*size*/) ATTRIBUTE_HIDDEN; diff --git a/bfd/lynx-core.c b/bfd/lynx-core.c index 44d94ad8745..9ec5a0d2028 100644 --- a/bfd/lynx-core.c +++ b/bfd/lynx-core.c @@ -96,7 +96,7 @@ lynx_core_file_p (bfd *abfd) asection *newsect; size_t amt; - pagesize = getpagesize (); /* Serious cross-target issue here... This + pagesize = _bfd_pagesize; /* Serious cross-target issue here... This really needs to come from a system-specific header file. */ diff --git a/bfd/opncls.c b/bfd/opncls.c index c764d204831..e6337b88e18 100644 --- a/bfd/opncls.c +++ b/bfd/opncls.c @@ -163,6 +163,18 @@ _bfd_new_bfd_contained_in (bfd *obfd) static void _bfd_delete_bfd (bfd *abfd) { +#ifdef USE_MMAP + struct bfd_mmapped *mmapped, *next; + for (mmapped = abfd->mmapped; mmapped != NULL; mmapped = next) + { + struct bfd_mmapped_entry *entries = mmapped->entries; + next = mmapped->next; + for (unsigned int i = 0; i < mmapped->next_entry; i++) + munmap (entries[i].addr, entries[i].size); + munmap (mmapped, _bfd_pagesize); + } +#endif + /* Give the target _bfd_free_cached_info a chance to free memory. */ if (abfd->memory && abfd->xvec) bfd_free_cached_info (abfd); diff --git a/bfd/sysdep.h b/bfd/sysdep.h index b907bc26a09..173f5804df3 100644 --- a/bfd/sysdep.h +++ b/bfd/sysdep.h @@ -98,6 +98,10 @@ #define MAP_FAILED ((void *) -1) #endif +#ifndef MAP_ANONYMOUS +#define MAP_ANONYMOUS MAP_ANON +#endif + #include "filenames.h" #if !HAVE_DECL_FFS From patchwork Sun Mar 17 12:19:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 87287 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 538F83858006 for ; Sun, 17 Mar 2024 12:20:43 +0000 (GMT) X-Original-To: binutils@sourceware.org Delivered-To: binutils@sourceware.org Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by sourceware.org (Postfix) with ESMTPS id 92FA53858C50 for ; Sun, 17 Mar 2024 12:19:17 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 92FA53858C50 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 92FA53858C50 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::631 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677961; cv=none; b=gR3a++5Ysh4ElwTLsekMmOg/ZBDFoLaLZ34RPcCUqNxyWaSTxSig2/+gZ3TF/Kmyur0D+qbcxsjGWeHrwmS/AVle2/m0EgxfWeg5wi9K+ETKE1jRPlS30EncrXs3pycmZqVQtSm9xxVjnhXtyq/606MLsf66yqi1qGsPR8jKxLM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677961; c=relaxed/simple; bh=l6Q3TdHwdJxAKkm38LAOui+l+2+KYO3+kFam8GL0UOs=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=G9PgeeBTBeW+tDAql294rSPwbTmB1K761XOnX6A97tQRwOXXi6NwdqtBsF0fxXe84rqK18in8ERBh6WGwy15ReBmSaNhXWxNlJml6mjSnRgYbhzqfpSZQUzxYOzXbJp5lCXSIHtcrVDTHR2OyUCRZ8Jm+LaNV3MHfqWniRxkbkY= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1def89f0cfdso11751585ad.0 for ; Sun, 17 Mar 2024 05:19:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710677956; x=1711282756; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wZNt6MzHkf/nIouE0CVmrspfEkJvLyg7ZqhGDPbZ8Oo=; b=YECxb8ri7eJ47WDV6KHZqAbDP2XV4JtsNWlqN+GocvLWAGDkEHFHCGBfluBAJfltPo 31NkqekNfDnW8POmndUxOkQzH8Bn7QgAc4sTahPzzqk8ilw+sXk+lL9JjifdCMdCvtLe y59FWyxHq4vdCYDQEowvFj5gPPFjZeoUtb7XlPd8O0702S5qFhcvRXd9P9K9SBnaYAkj mE59875qWdAWwKlRagZfL6eewP3K0JLZhOJYvGHG2ercU+KVPbaEnjokdJilMccTPx2G CMtSMZKOEAhTCSFoasS78EI4jx02vgXS17nChhpK74VzNmV2MbyMnZzjaCN8dHdo4QNE 3zqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710677956; x=1711282756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wZNt6MzHkf/nIouE0CVmrspfEkJvLyg7ZqhGDPbZ8Oo=; b=ZwGX13twT5SzYyWWeHR8MrYz/nIYHlFHjPJLyxek/DZTLY7a+ibNO4uo8kSyHX4HXz B+BPUJYxR0J5Ri0CrnVUBOFtDjMgQzQjy9D4Ls8eihN2uuzOCNN0N9nrDsB2OLvowO+x oP2yFNdS6Ek5v9mLqk26Xh3Yrje+sfboFRCGWR1KKJgQ2qFVSPMC4ZeuqwXwWBNqXZSJ FYhMl7vagTLOv/LZhdQrJdveG/Xag0tmZy7AutniTrOTqvadCsVqQz44I1WluZz0vc99 C01L0+dUNF/rZRWZq6NuOWbdICtNex/67+JZzaGQlz8P1tDHIZNIDiNm1nxp0jdumx4+ AMCA== X-Gm-Message-State: AOJu0YxiZ2kKszhWxxvh2lEAsEAihDzDI0dDosLJWRiijqI562hPafqb hHvKdtNA0TzIlDAbfw59cw3Gsx57FY7LdMnPZ9eqhnhzjwY2eHQR9W8LA4yZ X-Google-Smtp-Source: AGHT+IEQXyNRSiQXLYMvu5tCGHerxX3ASyCdRDXQS2Y+9+U59tWp9YunP9Q0hJbmF4xgDN+SSGm4Qw== X-Received: by 2002:a17:902:c206:b0:1e0:e36:5e45 with SMTP id 6-20020a170902c20600b001e00e365e45mr1940892pll.2.1710677956320; Sun, 17 Mar 2024 05:19:16 -0700 (PDT) Received: from gnu-cfl-3.localdomain ([172.58.89.72]) by smtp.gmail.com with ESMTPSA id q12-20020a17090311cc00b001db9e12cd62sm7152386plh.10.2024.03.17.05.19.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 17 Mar 2024 05:19:14 -0700 (PDT) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id E7E637406C5; Sun, 17 Mar 2024 05:19:12 -0700 (PDT) From: "H.J. Lu" To: binutils@sourceware.org Cc: goldstein.w.n@gmail.com, sam@gentoo.org, amodra@gmail.com Subject: [PATCH v12 2/6] elf: Add _bfd_elf_m[un]map_section_contents Date: Sun, 17 Mar 2024 05:19:08 -0700 Message-ID: <20240317121912.799372-3-hjl.tools@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240317121912.799372-1-hjl.tools@gmail.com> References: <20240317121912.799372-1-hjl.tools@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-3019.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_ABUSEAT, RCVD_IN_DNSWL_NONE, RCVD_IN_SBL_CSS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: binutils-bounces+patchwork=sourceware.org@sourceware.org Add _bfd_elf_mmap_section_contents and _bfd_elf_munmap_section_contents. A backend must opt-in to use mmap. It should replace bfd_malloc_and_get_section -> _bfd_elf_mmap_section_contents free -> _bfd_elf_munmap_section_contents on section contents. * compress.c (bfd_get_full_section_contents): Don't allocate buffer if mmapped_p is true. * elf-bfd.h (elf_backend_data): Add use_mmap. (bfd_elf_section_data): Add contents_addr and contents_size. (_bfd_elf_mmap_section_contents): New. (_bfd_elf_munmap_section_contents): Likewise. * elf-eh-frame.c (_bfd_elf_parse_eh_frame): Replace bfd_malloc_and_get_section and free with _bfd_elf_mmap_section_contents and _bfd_elf_munmap_section_contents on section contents. * elf-sframe.c (_bfd_elf_parse_sframe): Likewise. * elf.c (_bfd_elf_make_section_from_shdr): Replace bfd_malloc_and_get_section and free with _bfd_elf_mmap_section_contents and _bfd_elf_munmap_section_contents on section contents. (_bfd_elf_print_private_bfd_data): Likewise. (_bfd_elf_mmap_section_contents): New. (_bfd_elf_munmap_section_contents): Likewise. * elf32-i386.c (elf_i386_scan_relocs): Replace bfd_malloc_and_get_section and free with _bfd_elf_mmap_section_contents and _bfd_elf_munmap_section_contents on section contents. * elf64-x86-64.c (elf_x86_64_scan_relocs): Likewise. (elf_x86_64_get_synthetic_symtab): Likewise. * elfcode.h (elf_checksum_contents): Likewise. * elflink.c (elf_link_add_object_symbols): Likewise. (bfd_elf_get_bfd_needed_list): Likewise. * elfxx-target.h (elf_backend_use_mmap): New. (elfNN_bed): Add elf_backend_use_mmap. * elfxx-x86.c (elf_x86_size_or_finish_relative_reloc): Replace bfd_malloc_and_get_section and free with _bfd_elf_mmap_section_contents and _bfd_elf_munmap_section_contents on section contents. (_bfd_x86_elf_get_synthetic_symtab): Replace free with _bfd_elf_munmap_section_contents. * elfxx-x86.h (elf_backend_use_mmap): New. * libbfd.c: Include "elf-bfd.h". (_bfd_generic_get_section_contents): Call bfd_mmap_local for mmapped_p. * opncls.c (_bfd_delete_bfd): Also munmap ELF section contents. * section.c (asection): Add mmapped_p. (BFD_FAKE_SECTION): Updated. (bfd_malloc_and_get_section): Add a sanity check for not mmapped_p. * bfd-in2.h: Regenerated. --- bfd/bfd-in2.h | 7 ++-- bfd/compress.c | 2 +- bfd/elf-bfd.h | 20 +++++++++++ bfd/elf-eh-frame.c | 4 +-- bfd/elf-sframe.c | 4 +-- bfd/elf.c | 87 +++++++++++++++++++++++++++++++++++++++++++--- bfd/elf32-i386.c | 6 ++-- bfd/elf64-x86-64.c | 10 +++--- bfd/elfcode.h | 7 ++-- bfd/elflink.c | 12 +++---- bfd/elfxx-target.h | 6 +++- bfd/elfxx-x86.c | 8 ++--- bfd/elfxx-x86.h | 1 + bfd/libbfd.c | 59 +++++++++++++++++++++++++++++-- bfd/opncls.c | 9 +++++ bfd/section.c | 9 +++-- 16 files changed, 212 insertions(+), 39 deletions(-) diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h index 544422a9522..dc6bceb6a96 100644 --- a/bfd/bfd-in2.h +++ b/bfd/bfd-in2.h @@ -688,6 +688,9 @@ typedef struct bfd_section /* Nonzero if this section uses RELA relocations, rather than REL. */ unsigned int use_rela_p:1; + /* Nonzero if this section contents are mmapped, rather than malloced. */ + unsigned int mmapped_p:1; + /* Bits used by various backends. The generic code doesn't touch these fields. */ @@ -975,8 +978,8 @@ discarded_section (const asection *sec) /* linker_mark, linker_has_input, gc_mark, decompress_status, */ \ 0, 0, 1, 0, \ \ - /* segment_mark, sec_info_type, use_rela_p, */ \ - 0, 0, 0, \ + /* segment_mark, sec_info_type, use_rela_p, mmapped_p, */ \ + 0, 0, 0, 0, \ \ /* sec_flg0, sec_flg1, sec_flg2, sec_flg3, sec_flg4, sec_flg5, */ \ 0, 0, 0, 0, 0, 0, \ diff --git a/bfd/compress.c b/bfd/compress.c index 8bc44de813b..6c211843e60 100644 --- a/bfd/compress.c +++ b/bfd/compress.c @@ -749,7 +749,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) switch (compress_status) { case COMPRESS_SECTION_NONE: - if (p == NULL) + if (p == NULL && !sec->mmapped_p) { p = (bfd_byte *) bfd_malloc (allocsz); if (p == NULL) diff --git a/bfd/elf-bfd.h b/bfd/elf-bfd.h index c5d325435b6..e7c2cd19bed 100644 --- a/bfd/elf-bfd.h +++ b/bfd/elf-bfd.h @@ -1775,6 +1775,12 @@ struct elf_backend_data /* True if the 64-bit Linux PRPSINFO structure's `pr_uid' and `pr_gid' members use a 16-bit data type. */ unsigned linux_prpsinfo64_ugid16 : 1; + + /* True if the backend can use mmap to map in all input section + contents. All bfd_malloc_and_get_section and free usages on + section contents must be replaced by _bfd_elf_mmap_section_contents + and _bfd_elf_munmap_section_contents. */ + unsigned use_mmap : 1; }; /* Information about reloc sections associated with a bfd_elf_section_data @@ -1856,6 +1862,15 @@ struct bfd_elf_section_data /* Link from a text section to its .eh_frame_entry section. */ asection *eh_frame_entry; + /* If the mmapped_p flag is set, this points to the actual mmapped + address of contents. If it is set to NULL, contents isn't + mmapped. */ + void *contents_addr; + + /* If the mmapped_p flag is set, this is the actual mmapped size of + contents. */ + size_t contents_size; + /* TRUE if the section has secondary reloc sections associated with it. FIXME: In the future it might be better to change this into a list of secondary reloc sections, making lookup easier and faster. */ @@ -3124,6 +3139,11 @@ extern bool _bfd_elf_maybe_set_textrel extern bool _bfd_elf_add_dynamic_tags (bfd *, struct bfd_link_info *, bool); +extern bool _bfd_elf_mmap_section_contents + (bfd *abfd, asection *section, bfd_byte **buf); +extern void _bfd_elf_munmap_section_contents + (asection *, void *); + /* Large common section. */ extern asection _bfd_elf_large_com_section; diff --git a/bfd/elf-eh-frame.c b/bfd/elf-eh-frame.c index 9a504234163..902d7c16334 100644 --- a/bfd/elf-eh-frame.c +++ b/bfd/elf-eh-frame.c @@ -618,7 +618,7 @@ _bfd_elf_parse_eh_frame (bfd *abfd, struct bfd_link_info *info, /* Read the frame unwind information from abfd. */ - REQUIRE (bfd_malloc_and_get_section (abfd, sec, &ehbuf)); + REQUIRE (_bfd_elf_mmap_section_contents (abfd, sec, &ehbuf)); /* If .eh_frame section size doesn't fit into int, we cannot handle it (it would need to use 64-bit .eh_frame format anyway). */ @@ -1052,7 +1052,7 @@ _bfd_elf_parse_eh_frame (bfd *abfd, struct bfd_link_info *info, hdr_info->u.dwarf.table = false; free (sec_info); success: - free (ehbuf); + _bfd_elf_munmap_section_contents (sec, ehbuf); free (local_cies); #undef REQUIRE } diff --git a/bfd/elf-sframe.c b/bfd/elf-sframe.c index bfc875cd9fc..a4e3143e9d2 100644 --- a/bfd/elf-sframe.c +++ b/bfd/elf-sframe.c @@ -208,7 +208,7 @@ _bfd_elf_parse_sframe (bfd *abfd, } /* Read the SFrame stack trace information from abfd. */ - if (!bfd_malloc_and_get_section (abfd, sec, &sfbuf)) + if (!_bfd_elf_mmap_section_contents (abfd, sec, &sfbuf)) goto fail_no_free; /* Decode the buffer and keep decoded contents for later use. @@ -241,7 +241,7 @@ fail_no_free: abfd, sec); return false; success: - free (sfbuf); + _bfd_elf_munmap_section_contents (sec, sfbuf); return true; } diff --git a/bfd/elf.c b/bfd/elf.c index c80fff47b45..557c1ac5f4a 100644 --- a/bfd/elf.c +++ b/bfd/elf.c @@ -1123,12 +1123,12 @@ _bfd_elf_make_section_from_shdr (bfd *abfd, { bfd_byte *contents; - if (!bfd_malloc_and_get_section (abfd, newsect, &contents)) + if (!_bfd_elf_mmap_section_contents (abfd, newsect, &contents)) return false; elf_parse_notes (abfd, (char *) contents, hdr->sh_size, hdr->sh_offset, hdr->sh_addralign); - free (contents); + _bfd_elf_munmap_section_contents (newsect, contents); } if ((newsect->flags & SEC_ALLOC) != 0) @@ -1708,7 +1708,7 @@ _bfd_elf_print_private_bfd_data (bfd *abfd, void *farg) fprintf (f, _("\nDynamic Section:\n")); - if (!bfd_malloc_and_get_section (abfd, s, &dynbuf)) + if (!_bfd_elf_mmap_section_contents (abfd, s, &dynbuf)) goto error_return; elfsec = _bfd_elf_section_from_bfd_section (abfd, s); @@ -1830,7 +1830,7 @@ _bfd_elf_print_private_bfd_data (bfd *abfd, void *farg) fprintf (f, "\n"); } - free (dynbuf); + _bfd_elf_munmap_section_contents (s, dynbuf); dynbuf = NULL; } @@ -1887,7 +1887,7 @@ _bfd_elf_print_private_bfd_data (bfd *abfd, void *farg) return true; error_return: - free (dynbuf); + _bfd_elf_munmap_section_contents (s, dynbuf); return false; } @@ -14392,3 +14392,80 @@ _bfd_elf_write_secondary_reloc_section (bfd *abfd, asection *sec) return result; } + +/* Mmap in section contents. */ + +bool +_bfd_elf_mmap_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **buf) +{ +#ifdef USE_MMAP + const struct elf_backend_data *bed = get_elf_backend_data (abfd); + if (bed->use_mmap + && sec->compress_status == COMPRESS_SECTION_NONE + && (sec->flags & SEC_LINKER_CREATED) == 0) + { + /* Use mmap only if section size >= the minimum mmap section + size. */ + size_t readsz = bfd_get_section_limit_octets (abfd, sec); + size_t allocsz = bfd_get_section_alloc_size (abfd, sec); + if (readsz == allocsz && readsz >= _bfd_minimum_mmap_size) + { + if (sec->contents != NULL) + { + if (!sec->mmapped_p) + abort (); + *buf = sec->contents; + return true; + } + if (sec->mmapped_p) + abort (); + sec->mmapped_p = 1; + } + } +#endif + *buf = NULL; + bool ret = bfd_get_full_section_contents (abfd, sec, buf); + if (ret && sec->mmapped_p) + *buf = sec->contents; + return ret; +} + +/* Munmap section contents. */ + +void +_bfd_elf_munmap_section_contents (asection *sec ATTRIBUTE_UNUSED, + void *contents) +{ + /* NB: Since _bfd_elf_munmap_section_contents is called like free, + CONTENTS may be NULL. */ + if (contents == NULL) + return; + +#ifdef USE_MMAP + if (sec->mmapped_p) + { + /* _bfd_elf_mmap_section_contents may return the previously + mapped section contents. Munmap the section contents only + if they haven't been cached. */ + if (elf_section_data (sec)->this_hdr.contents == contents) + return; + + /* When _bfd_elf_mmap_section_contents returns CONTENTS as + malloced, CONTENTS_ADDR is set to NULL. */ + if (elf_section_data (sec)->contents_addr != NULL) + { + /* NB: CONTENTS_ADDR and CONTENTS_SIZE must be valid. */ + if (munmap (elf_section_data (sec)->contents_addr, + elf_section_data (sec)->contents_size) != 0) + abort (); + sec->mmapped_p = 0; + sec->contents = NULL; + elf_section_data (sec)->contents_addr = NULL; + elf_section_data (sec)->contents_size = 0; + return; + } + } +#endif + + free (contents); +} diff --git a/bfd/elf32-i386.c b/bfd/elf32-i386.c index 703a48c2c0a..0f963c373be 100644 --- a/bfd/elf32-i386.c +++ b/bfd/elf32-i386.c @@ -1499,7 +1499,7 @@ elf_i386_scan_relocs (bfd *abfd, /* Get the section contents. */ if (elf_section_data (sec)->this_hdr.contents != NULL) contents = elf_section_data (sec)->this_hdr.contents; - else if (!bfd_malloc_and_get_section (abfd, sec, &contents)) + else if (!_bfd_elf_mmap_section_contents (abfd, sec, &contents)) { sec->check_relocs_failed = 1; return false; @@ -1933,7 +1933,7 @@ elf_i386_scan_relocs (bfd *abfd, if (elf_section_data (sec)->this_hdr.contents != contents) { if (!converted && !_bfd_link_keep_memory (info)) - free (contents); + _bfd_elf_munmap_section_contents (sec, contents); else { /* Cache the section contents for elf_link_input_bfd if any @@ -1951,7 +1951,7 @@ elf_i386_scan_relocs (bfd *abfd, error_return: if (elf_section_data (sec)->this_hdr.contents != contents) - free (contents); + _bfd_elf_munmap_section_contents (sec, contents); sec->check_relocs_failed = 1; return false; } diff --git a/bfd/elf64-x86-64.c b/bfd/elf64-x86-64.c index 3300a2017bd..2e3f86840e5 100644 --- a/bfd/elf64-x86-64.c +++ b/bfd/elf64-x86-64.c @@ -2057,7 +2057,7 @@ elf_x86_64_scan_relocs (bfd *abfd, struct bfd_link_info *info, /* Get the section contents. */ if (elf_section_data (sec)->this_hdr.contents != NULL) contents = elf_section_data (sec)->this_hdr.contents; - else if (!bfd_malloc_and_get_section (abfd, sec, &contents)) + else if (!_bfd_elf_mmap_section_contents (abfd, sec, &contents)) { sec->check_relocs_failed = 1; return false; @@ -2591,7 +2591,7 @@ elf_x86_64_scan_relocs (bfd *abfd, struct bfd_link_info *info, if (elf_section_data (sec)->this_hdr.contents != contents) { if (!converted && !_bfd_link_keep_memory (info)) - free (contents); + _bfd_elf_munmap_section_contents (sec, contents); else { /* Cache the section contents for elf_link_input_bfd if any @@ -2609,7 +2609,7 @@ elf_x86_64_scan_relocs (bfd *abfd, struct bfd_link_info *info, error_return: if (elf_section_data (sec)->this_hdr.contents != contents) - free (contents); + _bfd_elf_munmap_section_contents (sec, contents); sec->check_relocs_failed = 1; return false; } @@ -5274,7 +5274,7 @@ elf_x86_64_get_synthetic_symtab (bfd *abfd, continue; /* Get the PLT section contents. */ - if (!bfd_malloc_and_get_section (abfd, plt, &plt_contents)) + if (!_bfd_elf_mmap_section_contents (abfd, plt, &plt_contents)) break; /* Check what kind of PLT it is. */ @@ -5367,7 +5367,7 @@ elf_x86_64_get_synthetic_symtab (bfd *abfd, if (plt_type == plt_unknown) { - free (plt_contents); + _bfd_elf_munmap_section_contents (plt, plt_contents); continue; } diff --git a/bfd/elfcode.h b/bfd/elfcode.h index 1e0784611bc..39dfe0ba234 100644 --- a/bfd/elfcode.h +++ b/bfd/elfcode.h @@ -1195,6 +1195,7 @@ elf_checksum_contents (bfd *abfd, Elf_Internal_Shdr i_shdr; Elf_External_Shdr x_shdr; bfd_byte *contents, *free_contents; + asection *sec = NULL; i_shdr = *i_shdrp[count]; i_shdr.sh_offset = 0; @@ -1210,8 +1211,6 @@ elf_checksum_contents (bfd *abfd, contents = i_shdr.contents; if (contents == NULL) { - asection *sec; - sec = bfd_section_from_elf_index (abfd, count); if (sec != NULL) { @@ -1220,7 +1219,7 @@ elf_checksum_contents (bfd *abfd, { /* Force rereading from file. */ sec->flags &= ~SEC_IN_MEMORY; - if (!bfd_malloc_and_get_section (abfd, sec, &free_contents)) + if (!_bfd_elf_mmap_section_contents (abfd, sec, &free_contents)) continue; contents = free_contents; } @@ -1229,7 +1228,7 @@ elf_checksum_contents (bfd *abfd, if (contents != NULL) { (*process) (contents, i_shdr.sh_size, arg); - free (free_contents); + _bfd_elf_munmap_section_contents (sec, free_contents); } } diff --git a/bfd/elflink.c b/bfd/elflink.c index 42029f29f7a..216c124b207 100644 --- a/bfd/elflink.c +++ b/bfd/elflink.c @@ -4426,10 +4426,10 @@ elf_link_add_object_symbols (bfd *abfd, struct bfd_link_info *info) unsigned int elfsec; unsigned long shlink; - if (!bfd_malloc_and_get_section (abfd, s, &dynbuf)) + if (!_bfd_elf_mmap_section_contents (abfd, s, &dynbuf)) { error_free_dyn: - free (dynbuf); + _bfd_elf_munmap_section_contents (s, dynbuf); goto error_return; } @@ -4535,7 +4535,7 @@ elf_link_add_object_symbols (bfd *abfd, struct bfd_link_info *info) elf_tdata (abfd)->is_pie = (dyn.d_un.d_val & DF_1_PIE) != 0; } - free (dynbuf); + _bfd_elf_munmap_section_contents (s, dynbuf); } /* DT_RUNPATH overrides DT_RPATH. Do _NOT_ bfd_release, as that @@ -8283,7 +8283,7 @@ bfd_elf_get_bfd_needed_list (bfd *abfd, if (s == NULL || s->size == 0 || (s->flags & SEC_HAS_CONTENTS) == 0) return true; - if (!bfd_malloc_and_get_section (abfd, s, &dynbuf)) + if (!_bfd_elf_mmap_section_contents (abfd, s, &dynbuf)) goto error_return; elfsec = _bfd_elf_section_from_bfd_section (abfd, s); @@ -8329,12 +8329,12 @@ bfd_elf_get_bfd_needed_list (bfd *abfd, } } - free (dynbuf); + _bfd_elf_munmap_section_contents (s, dynbuf); return true; error_return: - free (dynbuf); + _bfd_elf_munmap_section_contents (s, dynbuf); return false; } diff --git a/bfd/elfxx-target.h b/bfd/elfxx-target.h index 1e6992b5793..89e3d36adb2 100644 --- a/bfd/elfxx-target.h +++ b/bfd/elfxx-target.h @@ -148,6 +148,9 @@ #ifndef elf_backend_strtab_flags #define elf_backend_strtab_flags 0 #endif +#ifndef elf_backend_use_mmap +#define elf_backend_use_mmap false +#endif #define bfd_elfNN_bfd_debug_info_start _bfd_void_bfd #define bfd_elfNN_bfd_debug_info_end _bfd_void_bfd @@ -974,7 +977,8 @@ static const struct elf_backend_data elfNN_bed = elf_backend_extern_protected_data, elf_backend_always_renumber_dynsyms, elf_backend_linux_prpsinfo32_ugid16, - elf_backend_linux_prpsinfo64_ugid16 + elf_backend_linux_prpsinfo64_ugid16, + elf_backend_use_mmap }; /* Forward declaration for use when initialising alternative_target field. */ diff --git a/bfd/elfxx-x86.c b/bfd/elfxx-x86.c index 508fd771da3..b6904273c42 100644 --- a/bfd/elfxx-x86.c +++ b/bfd/elfxx-x86.c @@ -1566,9 +1566,9 @@ elf_x86_size_or_finish_relative_reloc = elf_section_data (sec)->this_hdr.contents; else { - if (!bfd_malloc_and_get_section (sec->owner, - sec, - &contents)) + if (!_bfd_elf_mmap_section_contents (sec->owner, + sec, + &contents)) info->callbacks->einfo /* xgettext:c-format */ (_("%F%P: %pB: failed to allocate memory for section `%pA'\n"), @@ -3789,7 +3789,7 @@ _bfd_x86_elf_get_synthetic_symtab (bfd *abfd, count = n; for (j = 0; plts[j].name != NULL; j++) - free (plts[j].contents); + _bfd_elf_munmap_section_contents (plts[j].sec, plts[j].contents); free (dynrelbuf); diff --git a/bfd/elfxx-x86.h b/bfd/elfxx-x86.h index b3af9b841ba..8680e3c29b5 100644 --- a/bfd/elfxx-x86.h +++ b/bfd/elfxx-x86.h @@ -960,6 +960,7 @@ extern void _bfd_x86_elf_link_report_relative_reloc _bfd_elf_x86_size_relative_relocs #define elf_backend_finish_relative_relocs \ _bfd_elf_x86_finish_relative_relocs +#define elf_backend_use_mmap true #define ELF_P_ALIGN ELF_MINPAGESIZE diff --git a/bfd/libbfd.c b/bfd/libbfd.c index a79c814a0dc..e5147a29d69 100644 --- a/bfd/libbfd.c +++ b/bfd/libbfd.c @@ -21,6 +21,7 @@ #include "sysdep.h" #include "bfd.h" +#include "elf-bfd.h" #include "libbfd.h" #include "objalloc.h" @@ -1196,6 +1197,19 @@ _bfd_generic_get_section_contents (bfd *abfd, return false; } +#ifdef USE_MMAP + if (section->mmapped_p + && (section->contents != NULL || location != NULL)) + { + _bfd_error_handler + /* xgettext:c-format */ + (_("%pB: mapped section %pA has non-NULL buffer"), + abfd, section); + bfd_set_error (bfd_error_invalid_operation); + return false; + } +#endif + sz = bfd_get_section_limit_octets (abfd, section); if (offset + count < count || offset + count > sz @@ -1208,8 +1222,49 @@ _bfd_generic_get_section_contents (bfd *abfd, return false; } - if (bfd_seek (abfd, section->filepos + offset, SEEK_SET) != 0 - || bfd_read (location, count, abfd) != count) + if (bfd_seek (abfd, section->filepos + offset, SEEK_SET) != 0) + return false; + +#ifdef USE_MMAP + if (section->mmapped_p) + { + if (location != 0 + || bfd_get_flavour (abfd) != bfd_target_elf_flavour) + abort (); + + int prot = ((section->reloc_count == 0) + ? PROT_READ : PROT_READ | PROT_WRITE); + + location = bfd_mmap_local + (abfd, count, prot, &elf_section_data (section)->contents_addr, + &elf_section_data (section)->contents_size); + + if (location == NULL) + return false; + + /* Check for iovec not supporting mmap. */ + if (location != MAP_FAILED) + { + section->contents = location; + return true; + } + + /* Malloc the buffer and call bfd_read. */ + location = (bfd_byte *) bfd_malloc (count); + if (location == NULL) + { + if (bfd_get_error () == bfd_error_no_memory) + _bfd_error_handler + /* xgettext:c-format */ + (_("error: %pB(%pA) is too large (%#" PRIx64 " bytes)"), + abfd, section, (uint64_t) count); + return false; + } + section->contents = location; + } +#endif + + if (bfd_read (location, count, abfd) != count) return false; return true; diff --git a/bfd/opncls.c b/bfd/opncls.c index e6337b88e18..5efec37175e 100644 --- a/bfd/opncls.c +++ b/bfd/opncls.c @@ -164,6 +164,15 @@ static void _bfd_delete_bfd (bfd *abfd) { #ifdef USE_MMAP + if (bfd_get_flavour (abfd) == bfd_target_elf_flavour) + { + asection *sec; + for (sec = abfd->sections; sec != NULL; sec = sec->next) + if (sec->mmapped_p) + munmap (elf_section_data (sec)->contents_addr, + elf_section_data (sec)->contents_size); + } + struct bfd_mmapped *mmapped, *next; for (mmapped = abfd->mmapped; mmapped != NULL; mmapped = next) { diff --git a/bfd/section.c b/bfd/section.c index 8cd30e80f2b..604105b39c4 100644 --- a/bfd/section.c +++ b/bfd/section.c @@ -422,6 +422,9 @@ CODE_FRAGMENT . {* Nonzero if this section uses RELA relocations, rather than REL. *} . unsigned int use_rela_p:1; . +. {* Nonzero if this section contents are mmapped, rather than malloced. *} +. unsigned int mmapped_p:1; +. . {* Bits used by various backends. The generic code doesn't touch . these fields. *} . @@ -711,8 +714,8 @@ EXTERNAL . {* linker_mark, linker_has_input, gc_mark, decompress_status, *} \ . 0, 0, 1, 0, \ . \ -. {* segment_mark, sec_info_type, use_rela_p, *} \ -. 0, 0, 0, \ +. {* segment_mark, sec_info_type, use_rela_p, mmapped_p, *} \ +. 0, 0, 0, 0, \ . \ . {* sec_flg0, sec_flg1, sec_flg2, sec_flg3, sec_flg4, sec_flg5, *} \ . 0, 0, 0, 0, 0, 0, \ @@ -1625,6 +1628,8 @@ DESCRIPTION bool bfd_malloc_and_get_section (bfd *abfd, sec_ptr sec, bfd_byte **buf) { + if (sec->mmapped_p) + abort (); *buf = NULL; return bfd_get_full_section_contents (abfd, sec, buf); } From patchwork Sun Mar 17 12:19:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 87285 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 085E3385829F for ; Sun, 17 Mar 2024 12:19:52 +0000 (GMT) X-Original-To: binutils@sourceware.org Delivered-To: binutils@sourceware.org Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) by sourceware.org (Postfix) with ESMTPS id C18D23858D37 for ; Sun, 17 Mar 2024 12:19:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C18D23858D37 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org C18D23858D37 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::52e ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677958; cv=none; b=FENmiMKyGvWMX7as0PeENGiZdR0M/gw9LX41Lg8LnTRwax7EvhUAVhxwxbexQ13UGLAqukOhjDzKNKww6eIMdEERzCglj98atjTEMPboPxibYezQCFPpl9iIdOcKWs5eIEj9FTOeKBbQdbC8vpO0OnEcxX/7vq9j0l9ZnrI9uoY= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677958; c=relaxed/simple; bh=cUif8EDRcOt0wrXByfEpDvZO73Rsmjt4BXHWCubQxwQ=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=UOxRSJhD+hjORpKjLwEBwX6vb7S4QdCSdX27OVw6m3XFjfUTMtdRCv9FCe4nr0qbbIjMyq8JKBjEmQx4c5vP7VmYneimkrFhPNrOZmNwAekofSZIvrcYW/H1DH73YnPBcssPfSZvwR7iPPDwlSus65vFdt77oAOVkgDqsOApcKc= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pg1-x52e.google.com with SMTP id 41be03b00d2f7-53fa455cd94so2545470a12.2 for ; Sun, 17 Mar 2024 05:19:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710677955; x=1711282755; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VNTyCjM0V5mSzAAU47ebhXmIyUQPdSaBs8ItU6F51fU=; b=TtMKNbGwJ5ZIKCltqyfwnjJuv2YAArKFLOn4eT6AzTXhCPWg4YgqJFUD/CpqXOp2kU LiK3u8sp40nZVAObAP706fwe/h6kyb3hbROYEUYFlMLV/OK/Eh/kAuT5zUm6hrVUnKVh xDeI5Tp+dqH5EGL/1eF3p580m2dmffM8K17OFOAfDt8/eCSlrFQXN9Z/ULxQY1bdhhcb aWBTy6ijT2EJIK4GnDLktoB7Ae1+/Q6+vz0w5cFJ5ecqClYMQ5MG9Vdj7MVX6QNv9MSD 1D2ytgeZPJa5Zv0rbFnKj/2ThrHn+XpPB6BlZCWg88hkQr+/rCzi8DH3gTDe5RHiP9vG zjRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710677955; x=1711282755; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VNTyCjM0V5mSzAAU47ebhXmIyUQPdSaBs8ItU6F51fU=; b=DdFPZQngwxYXN2yVo4J+0FP6xs5RATVOvuebgh3NFO+vb0skiJ2uUukQNwL83DcE4T n50tDN2pniIO3azoa/vrfeaLwPoo3YDsoR48whD3FOgJpFs61ue7JjaxDX9cBetDHySJ Ay5+YSqHWH+o0xvwrXCPQqNd9Mcdx9cy03SESLLdZTOXJdbyVja87d3VUFHGsl8HgVjn ROm90kiba9KGVx+URoIwoHsBhgdQ5hTG+8l+m3F8tsOondDhmyioZBiwjz2RXpPE4JqU oWmU4Ek5AeytN9J1BdPjJpOb1cjWBZPP4yoCBWcLUcZvzCqde+dyAi3msgc6vCT75Qbs vnYg== X-Gm-Message-State: AOJu0Yy1P9GWhS+J7mM7x57jrvcwThvHapHDWzqFR/tr7gQuqJBDLe2Q Lp8sF2ZYcRWy/+f4ig8zPcC74Ag3IuZj+9vB9yVnR/9jROf/uL+W X-Google-Smtp-Source: AGHT+IHtFVTWnwuTnz+cNGvpP0AYPuDui6QDR4fDVM9oUJ1yjpLM2yo6Q91Ja7kiqT7xU1LYrlx7uA== X-Received: by 2002:a17:902:f68b:b0:1dc:abf0:14ee with SMTP id l11-20020a170902f68b00b001dcabf014eemr12705761plg.22.1710677954640; Sun, 17 Mar 2024 05:19:14 -0700 (PDT) Received: from gnu-cfl-3.localdomain ([172.58.89.72]) by smtp.gmail.com with ESMTPSA id b17-20020a170902d51100b001d8f81ecea1sm4513348plg.172.2024.03.17.05.19.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 17 Mar 2024 05:19:14 -0700 (PDT) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id 026C974071D; Sun, 17 Mar 2024 05:19:13 -0700 (PDT) From: "H.J. Lu" To: binutils@sourceware.org Cc: goldstein.w.n@gmail.com, sam@gentoo.org, amodra@gmail.com Subject: [PATCH v12 3/6] elf: Use mmap to map in symbol and relocation tables Date: Sun, 17 Mar 2024 05:19:09 -0700 Message-ID: <20240317121912.799372-4-hjl.tools@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240317121912.799372-1-hjl.tools@gmail.com> References: <20240317121912.799372-1-hjl.tools@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-3019.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_ABUSEAT, RCVD_IN_DNSWL_NONE, RCVD_IN_SBL_CSS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: binutils-bounces+patchwork=sourceware.org@sourceware.org Add _bfd_mmap_read_temporary to mmap in symbol tables and relocations whose sizes >= 4 * page size. For the final link, allocate an external relocation buffer of 4 * page size to avoid using mmap and munmap on smaller relocation sections. Since _bfd_mmap_read_temporary allocates buffer as needed, its callers don't need to. When mmap is used to map in all ELF sections, data to link the 3.5GB clang executable in LLVM 17 debug build on Linux/x86-64 with 32GB RAM is: stdio mmap improvement user 84.79 85.27 -0.5% system 10.95 9.09 17% total 97.91 94.90 3% page faults 4837944 4033778 17% and data to link the 275M cc1plus executable in GCC 14 stage 1 build is: user 5.31 5.33 -0.4% system 0.86 0.76 12% total 6.19 6.13 1% page faults 361273 322491 11% * elf.c (bfd_elf_get_elf_syms): Don't allocate buffer for external symbol table. Replace bfd_read with _bfd_mmap_read_temporary. * elflink.c (elf_link_read_relocs_from_section): Add 2 arguments to return mmap memory address and size. (_bfd_elf_link_info_read_relocs): Don't allocate buffer for external relocation information. Replace bfd_read with _bfd_mmap_read_temporary. (bfd_elf_final_link): Cache external relocations up to _bfd_minimum_mmap_size bytes when mmap is used. * libbfd.c (_bfd_mmap_read_temporary): New. * libbfd-in.h (_bfd_mmap_read_temporary): Likewise. * libbfd.h: Regenerated. --- bfd/elf.c | 31 +++++++++++++---------------- bfd/elflink.c | 50 +++++++++++++++++++++++++---------------------- bfd/libbfd-in.h | 3 +++ bfd/libbfd.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++++ bfd/libbfd.h | 3 +++ 5 files changed, 98 insertions(+), 41 deletions(-) diff --git a/bfd/elf.c b/bfd/elf.c index 557c1ac5f4a..cb47c499704 100644 --- a/bfd/elf.c +++ b/bfd/elf.c @@ -460,19 +460,16 @@ bfd_elf_get_elf_syms (bfd *ibfd, goto out; } pos = symtab_hdr->sh_offset + symoffset * extsym_size; - if (extsym_buf == NULL) - { - alloc_ext = bfd_malloc (amt); - extsym_buf = alloc_ext; - } - if (extsym_buf == NULL - || bfd_seek (ibfd, pos, SEEK_SET) != 0 - || bfd_read (extsym_buf, amt, ibfd) != amt) + size_t alloc_ext_size = amt; + if (bfd_seek (ibfd, pos, SEEK_SET) != 0 + || !_bfd_mmap_read_temporary (&extsym_buf, &alloc_ext_size, + &alloc_ext, ibfd, false)) { intsym_buf = NULL; goto out; } + size_t alloc_extshndx_size = 0; if (shndx_hdr == NULL || shndx_hdr->sh_size == 0) extshndx_buf = NULL; else @@ -483,15 +480,13 @@ bfd_elf_get_elf_syms (bfd *ibfd, intsym_buf = NULL; goto out; } + alloc_extshndx_size = amt; pos = shndx_hdr->sh_offset + symoffset * sizeof (Elf_External_Sym_Shndx); - if (extshndx_buf == NULL) - { - alloc_extshndx = (Elf_External_Sym_Shndx *) bfd_malloc (amt); - extshndx_buf = alloc_extshndx; - } - if (extshndx_buf == NULL - || bfd_seek (ibfd, pos, SEEK_SET) != 0 - || bfd_read (extshndx_buf, amt, ibfd) != amt) + if (bfd_seek (ibfd, pos, SEEK_SET) != 0 + || !_bfd_mmap_read_temporary ((void **) &extshndx_buf, + &alloc_extshndx_size, + (void **) &alloc_extshndx, + ibfd, false)) { intsym_buf = NULL; goto out; @@ -530,8 +525,8 @@ bfd_elf_get_elf_syms (bfd *ibfd, } out: - free (alloc_ext); - free (alloc_extshndx); + _bfd_munmap_readonly_temporary (alloc_ext, alloc_ext_size); + _bfd_munmap_readonly_temporary (alloc_extshndx, alloc_extshndx_size); return intsym_buf; } diff --git a/bfd/elflink.c b/bfd/elflink.c index 216c124b207..9162df60bfa 100644 --- a/bfd/elflink.c +++ b/bfd/elflink.c @@ -2644,8 +2644,11 @@ _bfd_elf_link_assign_sym_version (struct elf_link_hash_entry *h, void *data) may be either a REL or a RELA section. The relocations are translated into RELA relocations and stored in INTERNAL_RELOCS, which should have already been allocated to contain enough space. - The EXTERNAL_RELOCS are a buffer where the external form of the - relocations should be stored. + The *EXTERNAL_RELOCS_P are a buffer where the external form of the + relocations should be stored. If *EXTERNAL_RELOCS_ADDR is NULL, + *EXTERNAL_RELOCS_ADDR and *EXTERNAL_RELOCS_SIZE returns the mmap + memory address and size. Otherwise, *EXTERNAL_RELOCS_ADDR is + unchanged and *EXTERNAL_RELOCS_SIZE returns 0. Returns FALSE if something goes wrong. */ @@ -2653,7 +2656,8 @@ static bool elf_link_read_relocs_from_section (bfd *abfd, asection *sec, Elf_Internal_Shdr *shdr, - void *external_relocs, + void **external_relocs_addr, + size_t *external_relocs_size, Elf_Internal_Rela *internal_relocs) { const struct elf_backend_data *bed; @@ -2663,13 +2667,17 @@ elf_link_read_relocs_from_section (bfd *abfd, Elf_Internal_Rela *irela; Elf_Internal_Shdr *symtab_hdr; size_t nsyms; + void *external_relocs = *external_relocs_addr; /* Position ourselves at the start of the section. */ if (bfd_seek (abfd, shdr->sh_offset, SEEK_SET) != 0) return false; /* Read the relocations. */ - if (bfd_read (external_relocs, shdr->sh_size, abfd) != shdr->sh_size) + *external_relocs_size = shdr->sh_size; + if (!_bfd_mmap_read_temporary (&external_relocs, + external_relocs_size, + external_relocs_addr, abfd, true)) return false; symtab_hdr = &elf_tdata (abfd)->symtab_hdr; @@ -2754,6 +2762,7 @@ _bfd_elf_link_info_read_relocs (bfd *abfd, bool keep_memory) { void *alloc1 = NULL; + size_t alloc1_size; Elf_Internal_Rela *alloc2 = NULL; const struct elf_backend_data *bed = get_elf_backend_data (abfd); struct bfd_elf_section_data *esdo = elf_section_data (o); @@ -2782,26 +2791,12 @@ _bfd_elf_link_info_read_relocs (bfd *abfd, goto error_return; } - if (external_relocs == NULL) - { - bfd_size_type size = 0; - - if (esdo->rel.hdr) - size += esdo->rel.hdr->sh_size; - if (esdo->rela.hdr) - size += esdo->rela.hdr->sh_size; - - alloc1 = bfd_malloc (size); - if (alloc1 == NULL) - goto error_return; - external_relocs = alloc1; - } - + alloc1 = external_relocs; internal_rela_relocs = internal_relocs; if (esdo->rel.hdr) { if (!elf_link_read_relocs_from_section (abfd, o, esdo->rel.hdr, - external_relocs, + &alloc1, &alloc1_size, internal_relocs)) goto error_return; external_relocs = (((bfd_byte *) external_relocs) @@ -2812,7 +2807,7 @@ _bfd_elf_link_info_read_relocs (bfd *abfd, if (esdo->rela.hdr && (!elf_link_read_relocs_from_section (abfd, o, esdo->rela.hdr, - external_relocs, + &alloc1, &alloc1_size, internal_rela_relocs))) goto error_return; @@ -2820,7 +2815,7 @@ _bfd_elf_link_info_read_relocs (bfd *abfd, if (keep_memory) esdo->relocs = internal_relocs; - free (alloc1); + _bfd_munmap_readonly_temporary (alloc1, alloc1_size); /* Don't free alloc2, since if it was allocated we are passing it back (under the name of internal_relocs). */ @@ -2828,7 +2823,7 @@ _bfd_elf_link_info_read_relocs (bfd *abfd, return internal_relocs; error_return: - free (alloc1); + _bfd_munmap_readonly_temporary (alloc1, alloc1_size); if (alloc2 != NULL) { if (keep_memory) @@ -12444,7 +12439,14 @@ bfd_elf_final_link (bfd *abfd, struct bfd_link_info *info) section, so that we know the sizes of the reloc sections. We also figure out some maximum sizes. */ max_contents_size = 0; +#ifdef USE_MMAP + /* Mmap is used only if section size >= the minimum mmap section + size. max_external_reloc_size covers all relocation sections + smaller than the minimum mmap section size. */ + max_external_reloc_size = _bfd_minimum_mmap_size; +#else max_external_reloc_size = 0; +#endif max_internal_reloc_count = 0; max_sym_count = 0; max_sym_shndx_count = 0; @@ -12533,8 +12535,10 @@ bfd_elf_final_link (bfd *abfd, struct bfd_link_info *info) if (esdi->rela.hdr != NULL) ext_size += esdi->rela.hdr->sh_size; +#ifndef USE_MMAP if (ext_size > max_external_reloc_size) max_external_reloc_size = ext_size; +#endif if (sec->reloc_count > max_internal_reloc_count) max_internal_reloc_count = sec->reloc_count; } diff --git a/bfd/libbfd-in.h b/bfd/libbfd-in.h index c5a79cf932c..889b221a950 100644 --- a/bfd/libbfd-in.h +++ b/bfd/libbfd-in.h @@ -905,6 +905,9 @@ extern void _bfd_munmap_readonly_temporary #define _bfd_munmap_readonly_temporary(ptr, rsize) free (ptr) #endif +extern bool _bfd_mmap_read_temporary + (void **, size_t *, void **, bfd *, bool) ATTRIBUTE_HIDDEN; + static inline void * _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) { diff --git a/bfd/libbfd.c b/bfd/libbfd.c index e5147a29d69..869f0ed5c66 100644 --- a/bfd/libbfd.c +++ b/bfd/libbfd.c @@ -1174,6 +1174,58 @@ _bfd_mmap_readonly_persistent (bfd *abfd, size_t rsize) } #endif +/* Attempt to read *SIZE_P bytes from ABFD's iostream to *DATA_P. + Return true if the full the amount has been read. If *DATA_P is + NULL, mmap should be used, return the memory address at the + current offset in *DATA_P as well as return mmap address and size + in *MMAP_BASE and *SIZE_P. Otherwise, return NULL in *MMAP_BASE + and 0 in *SIZE_P. If FINAL_LINK is true, this is called from + elf_link_read_relocs_from_section. */ + +bool +_bfd_mmap_read_temporary (void **data_p, size_t *size_p, + void **mmap_base, bfd *abfd, + bool final_link ATTRIBUTE_UNUSED) +{ + void *data = *data_p; + size_t size = *size_p; + +#ifdef USE_MMAP + /* NB: When FINAL_LINK is true, the size of the preallocated buffer + is _bfd_minimum_mmap_size and use mmap if the data size >= + _bfd_minimum_mmap_size. Otherwise, use mmap if ABFD isn't an IR + input or the data size >= _bfd_minimum_mmap_size. */ + bool use_mmmap; + bool mmap_size = size >= _bfd_minimum_mmap_size; + if (final_link) + use_mmmap = mmap_size; + else + use_mmmap = (mmap_size + && data == NULL + && (abfd->flags & BFD_PLUGIN) == 0); + if (use_mmmap) + { + data = _bfd_mmap_readonly_temporary (abfd, size, mmap_base, + size_p); + if (data == NULL || data == MAP_FAILED) + abort (); + *data_p = data; + return true; + } +#endif + + if (data == NULL) + { + data = bfd_malloc (size); + if (data == NULL) + return false; + *data_p = data; + } + *mmap_base = NULL; + *size_p = 0; + return bfd_read (data, size, abfd) == size; +} + /* Default implementation */ bool diff --git a/bfd/libbfd.h b/bfd/libbfd.h index 47f40889a95..876c1f223c7 100644 --- a/bfd/libbfd.h +++ b/bfd/libbfd.h @@ -911,6 +911,9 @@ extern void _bfd_munmap_readonly_temporary #define _bfd_munmap_readonly_temporary(ptr, rsize) free (ptr) #endif +extern bool _bfd_mmap_read_temporary + (void **, size_t *, void **, bfd *, bool) ATTRIBUTE_HIDDEN; + static inline void * _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) { From patchwork Sun Mar 17 12:19:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 87288 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3DB8438582A7 for ; Sun, 17 Mar 2024 12:21:15 +0000 (GMT) X-Original-To: binutils@sourceware.org Delivered-To: binutils@sourceware.org Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) by sourceware.org (Postfix) with ESMTPS id AD4B03858C52 for ; Sun, 17 Mar 2024 12:19:17 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org AD4B03858C52 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org AD4B03858C52 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::62c ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677960; cv=none; b=BSWkytliT3qufHIh5U6WaHM6mXx+HjxA8WJyGc25GmO5G8NIpkl4wcw9pTHTcHwIUkgNnXlGmqbyKZEn2w84Z8piPD/YEKqrnW1fssww75ecpMXHMtiHubbArYp/EBN75PQPD84kR68lGa/J+gs1iDhim/hlhkP6/sDjLqv4ZFI= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677960; c=relaxed/simple; bh=KLAYjAXvIyUlr1fUPbFWvshAulMgrfIbBZzi4OLvYBw=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=VGMrwHkVMLKQb1EahOp/xk/FoF1EIkitdxO0cnipsBEKam9DrCvFUcOjw9IhmxS1j/MYcsJ4Eyxf4DVHtS/vm4c3kVynQ8C77i0vhwa0P1FwjuaB7AjVFKL+EaInIN2fDxyXWexqL3SSourqczVNEgg0MEuk1fAFjMvfROwwoHY= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1e0025ef1efso4631795ad.1 for ; Sun, 17 Mar 2024 05:19:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710677957; x=1711282757; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qK/HQmr1FmS5tOtr7ybecZ3neE6WGAVVarU/uWEOa2w=; b=dAL4GgbmDYFHZujnfCzoDB2TZixgQ/ZPkRvw6Gce3nPcI0Tf1GKuLFelidDqNd8o8B umRi/nhGanw8BF90GMlvYNCUr+38SiWMQXqx/w9AkPTSakm5pKUtAeiu1KTyFN/6ZeST Ge2PD/5F915riwrUKyJivBVYN00QL+tUoc9/1hJrT2zaAiIyT0t3v6wXeNuXsdIgLRZg gfKRZSUflbkW7XrVTUZTodf41B7wezsn+NPfwK/MZNqeRgewKg1iXIwteixYTe7Tw7HF DPqgEwwJBSLIEl5FUegA0zN/jJ2qVONcECMLhKPwJvZA4yG1dM3WrlXPgJozJwituixp 9W0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710677957; x=1711282757; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qK/HQmr1FmS5tOtr7ybecZ3neE6WGAVVarU/uWEOa2w=; b=LHmmMoU3bj7QMmIwXvfztcS3AgdJgpVWxeMJ0+rHMGR7ilKCQTBHUrkL9z0+ZbIQec NumRWPbhUS1g12a0RmtOosHdUSOoMklcgVCcN6LNyt9Ymze/y64+Xp1OTWsZK7ReWZxe ohv62zABui6BqdF124Wy3wN5Fxk8OoXS7DZEawlI4MtXO0IvRjNSrRnorJ573YQ48jFS WQTsa+57EhkDaA/r29+0k0hnLEmEYkIJekxYvniptxkwkYOdn/8qc3sOgWk8ssMn4qZq w/G2gCJ0jm+RjvIi6J4lqamPhsSdIED3pg4BC9oW6CWbKmAUCLJc/T5XIde1rN0IWFoc NGfQ== X-Gm-Message-State: AOJu0YyVqHEZALCIdfhCZ9TRBReKuM/l2FSc0VL/pStnCpd0afXQ6I9c LK38slT9POh2sXeMsQyY8GIynM1M9ZkuMRpoNiE2HZqw4Z5KNW4X+mfIH/Te X-Google-Smtp-Source: AGHT+IFAHYAZahhr4w/rP9UdNontLZOqYWVCAcOm3jl02YHmbmQ/BFERDcQ6drLl/H2bx6XIDbh8CQ== X-Received: by 2002:a17:902:7798:b0:1dd:ae5b:86f1 with SMTP id o24-20020a170902779800b001ddae5b86f1mr8781607pll.29.1710677956565; Sun, 17 Mar 2024 05:19:16 -0700 (PDT) Received: from gnu-cfl-3.localdomain ([172.58.89.72]) by smtp.gmail.com with ESMTPSA id b11-20020a170902650b00b001d9aa663282sm6096806plk.266.2024.03.17.05.19.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 17 Mar 2024 05:19:14 -0700 (PDT) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id 0E2D2740722; Sun, 17 Mar 2024 05:19:13 -0700 (PDT) From: "H.J. Lu" To: binutils@sourceware.org Cc: goldstein.w.n@gmail.com, sam@gentoo.org, amodra@gmail.com Subject: [PATCH v12 4/6] elf: Don't cache symbol nor relocation tables with mmap Date: Sun, 17 Mar 2024 05:19:10 -0700 Message-ID: <20240317121912.799372-5-hjl.tools@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240317121912.799372-1-hjl.tools@gmail.com> References: <20240317121912.799372-1-hjl.tools@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-3019.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_ABUSEAT, RCVD_IN_DNSWL_NONE, RCVD_IN_SBL_CSS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: binutils-bounces+patchwork=sourceware.org@sourceware.org During a "-j 8" LLVM 17 debug build on a machine with 32GB RAM and 16GB swap, ld was killed by kernel because of out of memory: [79437.949336] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-9.scope,task=ld,pid=797431,uid=1000 [79437.949349] Out of memory: Killed process 797431 (ld) total-vm:9219600kB, anon-rss:6558156kB, file-rss:1792kB, shmem-rss:0kB, UID:1000 pgtables:17552kB oom_score_adj:0 Don't cache symbol nor relocation tables if they are mapped in. Data to link the 3.5GB clang executable in LLVM 17 debug build on Linux/x86-64 with 32GB RAM is: stdio mmap improvement user 86.73 87.02 -0.3% system 9.55 9.21 3.6% total 100.40 97.66 0.7% maximum set(GB) 17.34 13.14 24% page faults 4047667 3042877 25% and data to link the 275M cc1plus executable in GCC 14 stage 1 build is: user 5.41 5.44 -0.5% system 0.80 0.76 5% total 6.25 6.26 -0.2% maximum set(MB) 1323 968 27% page faults 323451 236371 27% These improve the overall system performance for parallel build by reducing memory usage and page faults. Also rename _bfd_link_keep_memory to _bfd_elf_link_keep_memory. Since the --no-keep-memory linker option causes: https://sourceware.org/bugzilla/show_bug.cgi?id=31458 this is opt-in by each backend. bfd/ * elf32-i386.c (elf_i386_scan_relocs): Remove _bfd_link_keep_memory. * elf64-x86-64.c (elf_x86_64_scan_relocs): Likewise. * elflink.c (_bfd_elf_link_keep_memory): New. (_bfd_elf_link_iterate_on_relocs): Replace _bfd_link_keep_memory with _bfd_elf_link_keep_memory. (elf_link_add_object_symbols): Likewise. (init_reloc_cookie): Likewise. (init_reloc_cookie_rels): Likewise. * libbfd-in.h (_bfd_link_keep_memory): Removed. * linker.c (_bfd_link_keep_memory): Likewise. * libbfd.h: Regenerated. --- bfd/elf32-i386.c | 2 +- bfd/elf64-x86-64.c | 2 +- bfd/elflink.c | 69 ++++++++++++++++++++++++++++++++++++++-------- bfd/libbfd-in.h | 3 -- bfd/libbfd.h | 3 -- bfd/linker.c | 35 ----------------------- 6 files changed, 59 insertions(+), 55 deletions(-) diff --git a/bfd/elf32-i386.c b/bfd/elf32-i386.c index 0f963c373be..8a6c0fa9060 100644 --- a/bfd/elf32-i386.c +++ b/bfd/elf32-i386.c @@ -1932,7 +1932,7 @@ elf_i386_scan_relocs (bfd *abfd, if (elf_section_data (sec)->this_hdr.contents != contents) { - if (!converted && !_bfd_link_keep_memory (info)) + if (!converted) _bfd_elf_munmap_section_contents (sec, contents); else { diff --git a/bfd/elf64-x86-64.c b/bfd/elf64-x86-64.c index 2e3f86840e5..7756c5b1744 100644 --- a/bfd/elf64-x86-64.c +++ b/bfd/elf64-x86-64.c @@ -2590,7 +2590,7 @@ elf_x86_64_scan_relocs (bfd *abfd, struct bfd_link_info *info, if (elf_section_data (sec)->this_hdr.contents != contents) { - if (!converted && !_bfd_link_keep_memory (info)) + if (!converted) _bfd_elf_munmap_section_contents (sec, contents); else { diff --git a/bfd/elflink.c b/bfd/elflink.c index 9162df60bfa..dcbe6e1a18c 100644 --- a/bfd/elflink.c +++ b/bfd/elflink.c @@ -49,6 +49,53 @@ struct elf_info_failed static bool _bfd_elf_fix_symbol_flags (struct elf_link_hash_entry *, struct elf_info_failed *); +/* Return false if linker should avoid caching relocation information + and symbol tables of input files in memory. */ + +static bool +_bfd_elf_link_keep_memory (struct bfd_link_info *info) +{ +#ifdef USE_MMAP + /* Don't cache symbol nor relocation tables if they are mapped in. + NB: Since the --no-keep-memory linker option causes: + + https://sourceware.org/bugzilla/show_bug.cgi?id=31458 + + this is opt-in by each backend. */ + const struct elf_backend_data *bed + = get_elf_backend_data (info->output_bfd); + if (bed->use_mmap) + return false; +#endif + bfd *abfd; + bfd_size_type size; + + if (!info->keep_memory) + return false; + + if (info->max_cache_size == (bfd_size_type) -1) + return true; + + abfd = info->input_bfds; + size = info->cache_size; + do + { + if (size >= info->max_cache_size) + { + /* Over the limit. Reduce the memory usage. */ + info->keep_memory = false; + return false; + } + if (!abfd) + break; + size += abfd->alloc_size; + abfd = abfd->link.next; + } + while (1); + + return true; +} + asection * _bfd_elf_section_for_symbol (struct elf_reloc_cookie *cookie, unsigned long r_symndx, @@ -4182,10 +4229,9 @@ _bfd_elf_link_iterate_on_relocs || bfd_is_abs_section (o->output_section)) continue; - internal_relocs = _bfd_elf_link_info_read_relocs (abfd, info, - o, NULL, - NULL, - _bfd_link_keep_memory (info)); + internal_relocs = _bfd_elf_link_info_read_relocs + (abfd, info, o, NULL, NULL, + _bfd_elf_link_keep_memory (info)); if (internal_relocs == NULL) return false; @@ -5551,10 +5597,9 @@ elf_link_add_object_symbols (bfd *abfd, struct bfd_link_info *info) && (s->flags & SEC_DEBUGGING) != 0)) continue; - internal_relocs = _bfd_elf_link_info_read_relocs (abfd, info, - s, NULL, - NULL, - _bfd_link_keep_memory (info)); + internal_relocs = _bfd_elf_link_info_read_relocs + (abfd, info, s, NULL, NULL, + _bfd_elf_link_keep_memory (info)); if (internal_relocs == NULL) goto error_free_vers; @@ -13614,7 +13659,7 @@ init_reloc_cookie (struct elf_reloc_cookie *cookie, info->callbacks->einfo (_("%P%X: can not read symbols: %E\n")); return false; } - if (_bfd_link_keep_memory (info) ) + if (_bfd_elf_link_keep_memory (info) ) { symtab_hdr->contents = (bfd_byte *) cookie->locsyms; info->cache_size += (cookie->locsymcount @@ -13651,9 +13696,9 @@ init_reloc_cookie_rels (struct elf_reloc_cookie *cookie, } else { - cookie->rels = _bfd_elf_link_info_read_relocs (abfd, info, sec, - NULL, NULL, - _bfd_link_keep_memory (info)); + cookie->rels = _bfd_elf_link_info_read_relocs + (abfd, info, sec, NULL, NULL, + _bfd_elf_link_keep_memory (info)); if (cookie->rels == NULL) return false; cookie->rel = cookie->rels; diff --git a/bfd/libbfd-in.h b/bfd/libbfd-in.h index 889b221a950..7bfc58f12d8 100644 --- a/bfd/libbfd-in.h +++ b/bfd/libbfd-in.h @@ -848,9 +848,6 @@ extern bfd_byte * _bfd_write_unsigned_leb128 extern struct bfd_link_info *_bfd_get_link_info (bfd *) ATTRIBUTE_HIDDEN; -extern bool _bfd_link_keep_memory (struct bfd_link_info *) - ATTRIBUTE_HIDDEN; - extern uintptr_t _bfd_pagesize ATTRIBUTE_HIDDEN; extern uintptr_t _bfd_pagesize_m1 ATTRIBUTE_HIDDEN; extern uintptr_t _bfd_minimum_mmap_size ATTRIBUTE_HIDDEN; diff --git a/bfd/libbfd.h b/bfd/libbfd.h index 876c1f223c7..36fe6c0ebd0 100644 --- a/bfd/libbfd.h +++ b/bfd/libbfd.h @@ -854,9 +854,6 @@ extern bfd_byte * _bfd_write_unsigned_leb128 extern struct bfd_link_info *_bfd_get_link_info (bfd *) ATTRIBUTE_HIDDEN; -extern bool _bfd_link_keep_memory (struct bfd_link_info *) - ATTRIBUTE_HIDDEN; - extern uintptr_t _bfd_pagesize ATTRIBUTE_HIDDEN; extern uintptr_t _bfd_pagesize_m1 ATTRIBUTE_HIDDEN; extern uintptr_t _bfd_minimum_mmap_size ATTRIBUTE_HIDDEN; diff --git a/bfd/linker.c b/bfd/linker.c index 36cca9624c2..eb42a78b622 100644 --- a/bfd/linker.c +++ b/bfd/linker.c @@ -3556,38 +3556,3 @@ _bfd_nolink_bfd_define_start_stop (struct bfd_link_info *info ATTRIBUTE_UNUSED, { return (struct bfd_link_hash_entry *) _bfd_ptr_bfd_null_error (sec->owner); } - -/* Return false if linker should avoid caching relocation infomation - and symbol tables of input files in memory. */ - -bool -_bfd_link_keep_memory (struct bfd_link_info * info) -{ - bfd *abfd; - bfd_size_type size; - - if (!info->keep_memory) - return false; - - if (info->max_cache_size == (bfd_size_type) -1) - return true; - - abfd = info->input_bfds; - size = info->cache_size; - do - { - if (size >= info->max_cache_size) - { - /* Over the limit. Reduce the memory usage. */ - info->keep_memory = false; - return false; - } - if (!abfd) - break; - size += abfd->alloc_size; - abfd = abfd->link.next; - } - while (1); - - return true; -} From patchwork Sun Mar 17 12:19:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 87290 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 33C2B3858426 for ; Sun, 17 Mar 2024 12:22:15 +0000 (GMT) X-Original-To: binutils@sourceware.org Delivered-To: binutils@sourceware.org Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by sourceware.org (Postfix) with ESMTPS id 28D153858402 for ; Sun, 17 Mar 2024 12:19:19 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 28D153858402 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 28D153858402 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::633 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677961; cv=none; b=xtC43fjeX+VuOLHdGeb1zUYK9qYQAHM9RdgAYbAQit41jkhtCq3MUeW32tAtzSjcXiZ/XizwD6ATTPcknHhWS/LMsLZ/kYvLBWGBJ2FbLH+RGalRfFYg6fTFrUTqL23yQ22ciGt328A0s+hoLq+3axAg1Zf0trH1YF1hUbd22eI= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677961; c=relaxed/simple; bh=MvL8soQye2pxrdrAiBempMnRCuSeLTuxLLbzefAxfb8=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=RGBk1u+G++mlZg2Ak4GMqGRu8sUR7WYiDU8wSnZgfJJaVMN0HVH4JyWRrV8dBBwEiF3B2giTYu7ULr3ioIab6LQiM85lmVEuyhBqBRZjAO4ebSoNOXahsuFuhlLjguVAh88Za8ayw6CGinZMbLzlnz0pBPZj/tETJLVaq91YXCU= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1dddad37712so33685885ad.3 for ; Sun, 17 Mar 2024 05:19:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710677958; x=1711282758; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bZNkny8eb+NNkPm3I+3YtoQHhrQiN11SMwzwh4QpAew=; b=IUREEbH4a1uxRKqNbgPfmKUbJ5Yejisk38qRQ7J3IdWPH4gmS5liVfQShvy6jMwKHU 9Dra1OnjcDmUruBsuJ/0L38FCh8U7VhoSoV//7oJwfS8GFXyRFH61OWX8l6z74W4pO3n AsqZQRG7ukRlWlL1ovoEOoLoV3CuB2BihF8ROLeAl78tdUeAvWb5KLQG9JFMhSRXPbnA CzsVI2Rwk/McHkwzuUmx+vcokkNXbIJkzux0MLlr2bXyOhpjqikYbgMHaufHbsxSZt1A Xy+Uzr4IfyTJR09BmSHBkPe1zDifEMxBxeWizX856zoF1vOk15nurzozexM4l/KK/k6e PG5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710677958; x=1711282758; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bZNkny8eb+NNkPm3I+3YtoQHhrQiN11SMwzwh4QpAew=; b=Xc5bLScJM9fdb+Z4wQTLGbhYIS/suAwaw4oC+1SMgaO4r9G5zzzj33R0OV0jGWAzSU O9uDXZ1tTksIz8FrGLl1Au3sYRWbm4LjokwVgiN/YRALCuvlz6lmVKgxAO9Y/g3NT4B5 Elutog5L+GTsijcO/kRU6dltEeDcioomqkOpKEGbhigFcKGLTp5XKa/3mg1kpDCW3wek K1VxJKfAey3fYA5br8ksrk3mAtURvARPbDGnLrGID6uhuFEza8BkV53VStYpEuxYEYFO VH5yi+3B5jOUBa9+qo9+Hs9PQORGQMdJjDBA7UtpRudLZsQc57pwmiFp1LZxtpRj52r2 4TaA== X-Gm-Message-State: AOJu0YzWQKBMkTxeNl1o5R3eOaXIQiv2lU5/wEuSE7GWID1rAykEf9CO PenbHqDohpcPaQbkkuXvhWKL4SyKKU+DZ8/VT7fhTTcMHSHfa7uT X-Google-Smtp-Source: AGHT+IGgmytEOiFpegYRym0ugBrVM9W17PmWB1jChckrLck/3+iuvzmOyhsMjNIjJ55dkoT253semw== X-Received: by 2002:a17:902:ba8c:b0:1dd:7485:b4c9 with SMTP id k12-20020a170902ba8c00b001dd7485b4c9mr9908136pls.22.1710677958066; Sun, 17 Mar 2024 05:19:18 -0700 (PDT) Received: from gnu-cfl-3.localdomain ([172.58.89.72]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b001ddb505d50asm7186521pll.244.2024.03.17.05.19.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 17 Mar 2024 05:19:16 -0700 (PDT) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id 1963C740723; Sun, 17 Mar 2024 05:19:13 -0700 (PDT) From: "H.J. Lu" To: binutils@sourceware.org Cc: goldstein.w.n@gmail.com, sam@gentoo.org, amodra@gmail.com Subject: [PATCH v12 5/6] elf: Always keep symbol table and relocation info for eh_frame Date: Sun, 17 Mar 2024 05:19:11 -0700 Message-ID: <20240317121912.799372-6-hjl.tools@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240317121912.799372-1-hjl.tools@gmail.com> References: <20240317121912.799372-1-hjl.tools@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-3019.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_ABUSEAT, RCVD_IN_DNSWL_NONE, RCVD_IN_SBL_CSS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: binutils-bounces+patchwork=sourceware.org@sourceware.org When --no-keep-memory is used, the symbol table and relocation info for eh_frame are freed after they are retrieved for each text section in the input object. If an input object has many text sections, the same data is retrieved and freed many times which can take a very long time. Update _bfd_elf_gc_mark to keep the symbol table and relocation info for eh_frame to avoid it. Data to link the 3.5GB clang executable in LLVM 17 debug build on Linux/x86-64 with 32GB RAM is: before after improvement user 86.31 86.44 -0.2% system 8.77 8.63 1.6% total 95.58 96.81 -1.3% maximum set(GB) 13.1 13.1 0% page faults 3024752 3028699 -1.3% and data to link the 275M cc1plus executable in GCC 14 stage 1 build is: user 5.49 5.46 -0.5% system 0.73 0.73 0% total 6.26 6.25 0.3% maximum set(MB) 964 964 0% page faults 235173 235796 -0.3% The memory usage impact is minimum and the link time of the Rust binary in https://sourceware.org/bugzilla/show_bug.cgi?id=31466 is reduced from 500+ seconds to 1.44 seconds, a 300x speedup. PR ld/31466 * elflink.c (init_reloc_cookie): Add a bool argument, keep_memory, for keeping memory. Always keep memory if keep_memory is true. (init_reloc_cookie_rels): Likewise (init_reloc_cookie_for_section): Add a bool argument for keeping memory and pass it to init_reloc_cookie and init_reloc_cookie_rels. (_bfd_elf_gc_mark_reloc): Pass false to _bfd_elf_gc_mark. (_bfd_elf_gc_mark): Pass true to init_reloc_cookie_for_section for the eh_frame section. Pass false to init_reloc_cookie_for_section for other sections. (_bfd_elf_gc_mark_extra_sections): Add Add a bool argument for keeping memory and pass it to _bfd_elf_gc_mark. (bfd_elf_parse_eh_frame_entries): Pass false to init_reloc_cookie and init_reloc_cookie_rels. (bfd_elf_gc_sections): Pass false to init_reloc_cookie_for_section and _bfd_elf_gc_mark. (bfd_elf_discard_info): Pass false to init_reloc_cookie_for_section and init_reloc_cookie. --- bfd/elflink.c | 43 +++++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 16 deletions(-) diff --git a/bfd/elflink.c b/bfd/elflink.c index dcbe6e1a18c..601d15e9cef 100644 --- a/bfd/elflink.c +++ b/bfd/elflink.c @@ -13621,7 +13621,8 @@ bfd_elf_final_link (bfd *abfd, struct bfd_link_info *info) static bool init_reloc_cookie (struct elf_reloc_cookie *cookie, - struct bfd_link_info *info, bfd *abfd) + struct bfd_link_info *info, bfd *abfd, + bool keep_memory) { Elf_Internal_Shdr *symtab_hdr; const struct elf_backend_data *bed; @@ -13659,7 +13660,7 @@ init_reloc_cookie (struct elf_reloc_cookie *cookie, info->callbacks->einfo (_("%P%X: can not read symbols: %E\n")); return false; } - if (_bfd_elf_link_keep_memory (info) ) + if (keep_memory || _bfd_elf_link_keep_memory (info)) { symtab_hdr->contents = (bfd_byte *) cookie->locsyms; info->cache_size += (cookie->locsymcount @@ -13687,7 +13688,7 @@ fini_reloc_cookie (struct elf_reloc_cookie *cookie, bfd *abfd) static bool init_reloc_cookie_rels (struct elf_reloc_cookie *cookie, struct bfd_link_info *info, bfd *abfd, - asection *sec) + asection *sec, bool keep_memory) { if (sec->reloc_count == 0) { @@ -13698,7 +13699,7 @@ init_reloc_cookie_rels (struct elf_reloc_cookie *cookie, { cookie->rels = _bfd_elf_link_info_read_relocs (abfd, info, sec, NULL, NULL, - _bfd_elf_link_keep_memory (info)); + keep_memory || _bfd_elf_link_keep_memory (info)); if (cookie->rels == NULL) return false; cookie->rel = cookie->rels; @@ -13724,11 +13725,12 @@ fini_reloc_cookie_rels (struct elf_reloc_cookie *cookie, static bool init_reloc_cookie_for_section (struct elf_reloc_cookie *cookie, struct bfd_link_info *info, - asection *sec) + asection *sec, bool keep_memory) { - if (!init_reloc_cookie (cookie, info, sec->owner)) + if (!init_reloc_cookie (cookie, info, sec->owner, keep_memory)) goto error1; - if (!init_reloc_cookie_rels (cookie, info, sec->owner, sec)) + if (!init_reloc_cookie_rels (cookie, info, sec->owner, sec, + keep_memory)) goto error2; return true; @@ -13939,7 +13941,7 @@ _bfd_elf_gc_mark (struct bfd_link_info *info, { struct elf_reloc_cookie cookie; - if (!init_reloc_cookie_for_section (&cookie, info, sec)) + if (!init_reloc_cookie_for_section (&cookie, info, sec, false)) ret = false; else { @@ -13957,7 +13959,14 @@ _bfd_elf_gc_mark (struct bfd_link_info *info, { struct elf_reloc_cookie cookie; - if (!init_reloc_cookie_for_section (&cookie, info, eh_frame)) + /* NB: When --no-keep-memory is used, the symbol table and + relocation info for eh_frame are freed after they are retrieved + for each text section in the input object. If an input object + has many text sections, the same data is retrieved and freed + many times which can take a very long time. Always keep the + symbol table and relocation info for eh_frame to avoid it. */ + if (!init_reloc_cookie_for_section (&cookie, info, eh_frame, + true)) ret = false; else { @@ -14397,13 +14406,14 @@ bfd_elf_parse_eh_frame_entries (bfd *abfd ATTRIBUTE_UNUSED, if (sec == NULL || sec->sec_info_type == SEC_INFO_TYPE_JUST_SYMS) continue; - if (!init_reloc_cookie (&cookie, info, ibfd)) + if (!init_reloc_cookie (&cookie, info, ibfd, false)) return false; for (sec = ibfd->sections; sec; sec = sec->next) { if (startswith (bfd_section_name (sec), ".eh_frame_entry") - && init_reloc_cookie_rels (&cookie, info, ibfd, sec)) + && init_reloc_cookie_rels (&cookie, info, ibfd, sec, + false)) { _bfd_elf_parse_eh_frame_entry (info, sec, &cookie); fini_reloc_cookie_rels (&cookie, sec); @@ -14448,7 +14458,8 @@ bfd_elf_gc_sections (bfd *abfd, struct bfd_link_info *info) if (sec == NULL || sec->sec_info_type == SEC_INFO_TYPE_JUST_SYMS) continue; sec = bfd_get_section_by_name (sub, ".eh_frame"); - while (sec && init_reloc_cookie_for_section (&cookie, info, sec)) + while (sec && init_reloc_cookie_for_section (&cookie, info, sec, + false)) { _bfd_elf_parse_eh_frame (sub, info, sec, &cookie); if (elf_section_data (sec)->sec_info @@ -14960,7 +14971,7 @@ bfd_elf_discard_info (bfd *output_bfd, struct bfd_link_info *info) if (bfd_get_flavour (abfd) != bfd_target_elf_flavour) continue; - if (!init_reloc_cookie_for_section (&cookie, info, i)) + if (!init_reloc_cookie_for_section (&cookie, info, i, false)) return -1; if (_bfd_discard_section_stabs (abfd, i, @@ -14991,7 +15002,7 @@ bfd_elf_discard_info (bfd *output_bfd, struct bfd_link_info *info) if (bfd_get_flavour (abfd) != bfd_target_elf_flavour) continue; - if (!init_reloc_cookie_for_section (&cookie, info, i)) + if (!init_reloc_cookie_for_section (&cookie, info, i, false)) return -1; _bfd_elf_parse_eh_frame (abfd, info, i, &cookie); @@ -15056,7 +15067,7 @@ bfd_elf_discard_info (bfd *output_bfd, struct bfd_link_info *info) if (bfd_get_flavour (abfd) != bfd_target_elf_flavour) continue; - if (!init_reloc_cookie_for_section (&cookie, info, i)) + if (!init_reloc_cookie_for_section (&cookie, info, i, false)) return -1; if (_bfd_elf_parse_sframe (abfd, info, i, &cookie)) @@ -15092,7 +15103,7 @@ bfd_elf_discard_info (bfd *output_bfd, struct bfd_link_info *info) if (bed->elf_backend_discard_info != NULL) { - if (!init_reloc_cookie (&cookie, info, abfd)) + if (!init_reloc_cookie (&cookie, info, abfd, false)) return -1; if ((*bed->elf_backend_discard_info) (abfd, &cookie, info)) From patchwork Sun Mar 17 12:19:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 87289 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 50F713858430 for ; Sun, 17 Mar 2024 12:21:33 +0000 (GMT) X-Original-To: binutils@sourceware.org Delivered-To: binutils@sourceware.org Received: from mail-pg1-x530.google.com (mail-pg1-x530.google.com [IPv6:2607:f8b0:4864:20::530]) by sourceware.org (Postfix) with ESMTPS id D97BB3858C62 for ; Sun, 17 Mar 2024 12:19:18 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D97BB3858C62 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org D97BB3858C62 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::530 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677961; cv=none; b=eIWUjNkoUrGqbxLlEeHlVjzq8N1tTw/k75WEgsZPiuUBpApuN/CPfT/FYFjfVxC/PyNWOnJuiado/zLPeiYpEvGy4xKK+NR4/luPpXV6v3f9Ux/OtW01AeQuC2jBz4PZFgE4GSyVC8zuRLUYEdU0opQYXIsWK8KQjWs8qNwa/tw= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677961; c=relaxed/simple; bh=5RRdOch05bqpG1wrC1MKhqLW8ZJDwkraxDFnl5QgbuU=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=lxvvzFwhKeT8S2mWYOvdUWI2yqD2xwmyjXWgxKzI9vKRAjdi9C1UYUqGoz2jBU0/0IdXo/SBW7MQ8xHnUPk/3YMTXS0B0yxY5dxZfyjA5FHlKSE2GImvw1f7YHkJYQ5BVfE1w0v0sRmASzjk/XCJkSvrL0otiZceMxp0sl30G/4= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-5dbf7b74402so2457271a12.0 for ; Sun, 17 Mar 2024 05:19:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710677958; x=1711282758; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eAxiAZU9DzlTdBTcH2XGmQrzBruveyasPofjyAXWJRw=; b=V67nc/9paYf1NVQzkLi5/htME5Rtak74TWFEEldOZz1j2uWdvNpfcHWbphYD6TpPAr mSEbegoMCQjGk18LjVrJmA4czXiNmVNFfPy5hyf6zVklDbZLOvficfYrMoKFpoGIkGUQ pK4rAsFlnIifMzYgAOHEh0v7t6egHFe/wWrBb6hXzGzgI2HHGyGtFNt4q9omv2AThaNZ n7acZrZfYN70O+OmE9+nK0yRLVDqI8q3pzIqTRCzK35q8i6ifd9QSZuM+ivHRmTgEsls NgDEBZrBnF71bsUF8WXM//DfWqrJsfjNUJvfwHn9lDEyyMQu7GHKTVtlkL0ujS29/J9b ngZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710677958; x=1711282758; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eAxiAZU9DzlTdBTcH2XGmQrzBruveyasPofjyAXWJRw=; b=ktLPAYRw3DB+kb4mvxv1U9wpwGbNWXw+OuyTdPPx6pGVMcV0zb4rnpn8kb4dC4+k8O TcGTk7mwpn9Wn69VsxtMmSF/t+UOFn0oNhw+HGZKtDI+NI36fRb7G2PbZV4hCVbhliKj OL64YoJRpPJr08v+pKqxpYwefxY2RBXBgzBkz/8iztcGWTqsjftUWR3smhjAm9U9y84q GUhCijezV/k9oiSriovQ6Clbu0DFvPMmOGjlDkkDJJwhLcZTWY2q5WpywKobdVHZWDJ6 tPlgKE0zTgykzQI7bIDngA06+ts4Kv2S19ssSdaap0JdCAUOfB0qkjrJDtHIzh9izw+t r5QA== X-Gm-Message-State: AOJu0YysArV4Cn8y/ACfxBcLYJjUWwNRzJNGqABnaDM8ymvM79HkWYvM FptEsDh/QQxvbC1QYC0FWrQYauVF6NcLJzaeFNh0OkILwUqX+WKQJ7LpMmIE X-Google-Smtp-Source: AGHT+IE8Te3/wJ50F2j2QsQav79/GKPJa2WRXztQgoV3J97twmkbM2ywErVoJguBhHCjRKM8Bt6Rww== X-Received: by 2002:a17:903:41d0:b0:1e0:119e:f935 with SMTP id u16-20020a17090341d000b001e0119ef935mr1655929ple.15.1710677957793; Sun, 17 Mar 2024 05:19:17 -0700 (PDT) Received: from gnu-cfl-3.localdomain ([172.58.89.72]) by smtp.gmail.com with ESMTPSA id v7-20020a170902b7c700b001dcf7d03824sm7148091plz.55.2024.03.17.05.19.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 17 Mar 2024 05:19:16 -0700 (PDT) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id 247DF740724; Sun, 17 Mar 2024 05:19:13 -0700 (PDT) From: "H.J. Lu" To: binutils@sourceware.org Cc: goldstein.w.n@gmail.com, sam@gentoo.org, amodra@gmail.com Subject: [PATCH v12 6/6] elf: Add _bfd_elf_link_m[un]map_section_contents Date: Sun, 17 Mar 2024 05:19:12 -0700 Message-ID: <20240317121912.799372-7-hjl.tools@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240317121912.799372-1-hjl.tools@gmail.com> References: <20240317121912.799372-1-hjl.tools@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-3019.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_ABUSEAT, RCVD_IN_DNSWL_NONE, RCVD_IN_SBL_CSS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: binutils-bounces+patchwork=sourceware.org@sourceware.org To copy input section contents, add _bfd_elf_link_mmap_section_contents and _bfd_elf_link_munmap_section_contents to mmap in the input sections. * elf-bfd.h (_bfd_elf_link_mmap_section_contents): New. (_bfd_elf_link_munmap_section_contents): Likewise. * elf.c (elf_mmap_section_contents): New. (_bfd_elf_mmap_section_contents): Use it. (_bfd_elf_link_mmap_section_contents): New. (_bfd_elf_link_munmap_section_contents): Likewise. * elflink.c (elf_link_input_bfd): Call _bfd_elf_link_mmap_section_contents instead of bfd_get_full_section_contents. Call _bfd_elf_link_munmap_section_contents to munmap the section contents. (bfd_elf_final_link): When mmap is used, initialize max_contents_size to _bfd_minimum_mmap_size and increase it for compressed or linker created sections or sections whose rawsize != size. --- bfd/elf-bfd.h | 4 ++++ bfd/elf.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++---- bfd/elflink.c | 33 +++++++++++++++++++++-------- 3 files changed, 82 insertions(+), 12 deletions(-) diff --git a/bfd/elf-bfd.h b/bfd/elf-bfd.h index e7c2cd19bed..0fcef04af36 100644 --- a/bfd/elf-bfd.h +++ b/bfd/elf-bfd.h @@ -3143,6 +3143,10 @@ extern bool _bfd_elf_mmap_section_contents (bfd *abfd, asection *section, bfd_byte **buf); extern void _bfd_elf_munmap_section_contents (asection *, void *); +extern bool _bfd_elf_link_mmap_section_contents + (bfd *abfd, asection *section, bfd_byte **buf); +extern void _bfd_elf_link_munmap_section_contents + (asection *); /* Large common section. */ extern asection _bfd_elf_large_com_section; diff --git a/bfd/elf.c b/bfd/elf.c index cb47c499704..f66ac9289df 100644 --- a/bfd/elf.c +++ b/bfd/elf.c @@ -14388,10 +14388,12 @@ _bfd_elf_write_secondary_reloc_section (bfd *abfd, asection *sec) return result; } -/* Mmap in section contents. */ +/* Mmap in section contents. If FINAL_LINK is false, set *BUF to NULL + before calling bfd_get_full_section_contents. */ -bool -_bfd_elf_mmap_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **buf) +static bool +elf_mmap_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **buf, + bool final_link) { #ifdef USE_MMAP const struct elf_backend_data *bed = get_elf_backend_data (abfd); @@ -14415,16 +14417,41 @@ _bfd_elf_mmap_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **buf) if (sec->mmapped_p) abort (); sec->mmapped_p = 1; + + /* Never use the preallocated buffer if mmapp is used. */ + *buf = NULL; } } #endif - *buf = NULL; + /* NB: When this is called from elf_link_input_bfd, FINAL_LINK is + true. If FINAL_LINK is false, *BUF is set to the preallocated + buffer if USE_MMAP is undefined and *BUF is set to NULL if + USE_MMAP is defined. */ + if (!final_link) + *buf = NULL; bool ret = bfd_get_full_section_contents (abfd, sec, buf); if (ret && sec->mmapped_p) *buf = sec->contents; return ret; } +/* Mmap in section contents. */ + +bool +_bfd_elf_mmap_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **buf) +{ + return elf_mmap_section_contents (abfd, sec, buf, false); +} + +/* Mmap in the full section contents for the final link. */ + +bool +_bfd_elf_link_mmap_section_contents (bfd *abfd, sec_ptr sec, + bfd_byte **buf) +{ + return elf_mmap_section_contents (abfd, sec, buf, true); +} + /* Munmap section contents. */ void @@ -14464,3 +14491,25 @@ _bfd_elf_munmap_section_contents (asection *sec ATTRIBUTE_UNUSED, free (contents); } + +/* Munmap the full section contents for the final link. */ + +void +_bfd_elf_link_munmap_section_contents (asection *sec ATTRIBUTE_UNUSED) +{ +#ifdef USE_MMAP + if (sec->mmapped_p && elf_section_data (sec)->contents_addr != NULL) + { + /* When _bfd_elf_link_mmap_section_contents returns CONTENTS as + malloced, CONTENTS_ADDR is set to NULL. */ + /* NB: CONTENTS_ADDR and CONTENTS_SIZE must be valid. */ + if (munmap (elf_section_data (sec)->contents_addr, + elf_section_data (sec)->contents_size) != 0) + abort (); + sec->mmapped_p = 0; + sec->contents = NULL; + elf_section_data (sec)->contents_addr = NULL; + elf_section_data (sec)->contents_size = 0; + } +#endif +} diff --git a/bfd/elflink.c b/bfd/elflink.c index 601d15e9cef..ab3fa308d44 100644 --- a/bfd/elflink.c +++ b/bfd/elflink.c @@ -11487,7 +11487,8 @@ elf_link_input_bfd (struct elf_final_link_info *flinfo, bfd *input_bfd) else { contents = flinfo->contents; - if (! bfd_get_full_section_contents (input_bfd, o, &contents)) + if (! _bfd_elf_link_mmap_section_contents (input_bfd, o, + &contents)) return false; } @@ -12045,6 +12046,9 @@ elf_link_input_bfd (struct elf_final_link_info *flinfo, bfd *input_bfd) } break; } + + /* Munmap the section contents for each input section. */ + _bfd_elf_link_munmap_section_contents (o); } return true; @@ -12483,13 +12487,17 @@ bfd_elf_final_link (bfd *abfd, struct bfd_link_info *info) /* Count up the number of relocations we will output for each output section, so that we know the sizes of the reloc sections. We also figure out some maximum sizes. */ - max_contents_size = 0; #ifdef USE_MMAP /* Mmap is used only if section size >= the minimum mmap section - size. max_external_reloc_size covers all relocation sections - smaller than the minimum mmap section size. */ + size. The initial max_contents_size value covers all sections + smaller than the minimum mmap section size. It may be increased + for compressed or linker created sections or sections whose + rawsize != size. max_external_reloc_size covers all relocation + sections smaller than the minimum mmap section size. */ + max_contents_size = _bfd_minimum_mmap_size; max_external_reloc_size = _bfd_minimum_mmap_size; #else + max_contents_size = 0; max_external_reloc_size = 0; #endif max_internal_reloc_count = 0; @@ -12525,10 +12533,19 @@ bfd_elf_final_link (bfd *abfd, struct bfd_link_info *info) if (sec->flags & SEC_MERGE) merged = true; - if (sec->rawsize > max_contents_size) - max_contents_size = sec->rawsize; - if (sec->size > max_contents_size) - max_contents_size = sec->size; +#ifdef USE_MMAP + /* Mmap is used only on non-compressed, non-linker created + sections whose rawsize == size. */ + if (sec->compress_status != COMPRESS_SECTION_NONE + || (sec->flags & SEC_LINKER_CREATED) != 0 + || sec->rawsize != sec->size) +#endif + { + if (sec->rawsize > max_contents_size) + max_contents_size = sec->rawsize; + if (sec->size > max_contents_size) + max_contents_size = sec->size; + } if (bfd_get_flavour (sec->owner) == bfd_target_elf_flavour && (sec->owner->flags & DYNAMIC) == 0)