From patchwork Sat Sep 3 01:23:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aaron Merey X-Patchwork-Id: 57309 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 0E22C385840F for ; Sat, 3 Sep 2022 01:23:56 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 0E22C385840F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1662168236; bh=mzBQWd3z28Q4YcWcWhlKbwYqdFFFQ6Wn1C9YpBIOeGs=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Help: List-Subscribe:From:Reply-To:From; b=WNCYRlmqhkDHyhEDYvd8o6ZhSU6TQD6zLwW7SYZCvZaZ/ILazG2JqzbswgA12xbqL jW/9ksEJ2jR1SE/kAFbem1s39V2jiVPRt6y35wozxfAc/gHV/lWtxkDsNsBPZ/Z5Lb dQEz+fu9EiEIT6nZR0nqcUeVJGdc91O/QWmADZZY= X-Original-To: elfutils-devel@sourceware.org Delivered-To: elfutils-devel@sourceware.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 8F7A33858D1E for ; Sat, 3 Sep 2022 01:23:44 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 8F7A33858D1E Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-440-mVtRbsOEPDusaeNEnEMWhg-1; Fri, 02 Sep 2022 21:23:43 -0400 X-MC-Unique: mVtRbsOEPDusaeNEnEMWhg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CE9BD801231 for ; Sat, 3 Sep 2022 01:23:42 +0000 (UTC) Received: from localhost.localdomain (unknown [10.22.32.33]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9B39C1121314; Sat, 3 Sep 2022 01:23:42 +0000 (UTC) To: elfutils-devel@sourceware.org Subject: [PATCH] debuginfod: Use auto-sized connection pool when -C is not given with arg Date: Fri, 2 Sep 2022 21:23:32 -0400 Message-Id: <20220903012332.1456366-1-amerey@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: elfutils-devel@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Elfutils-devel mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-Patchwork-Original-From: Aaron Merey via Elfutils-devel From: Aaron Merey Reply-To: Aaron Merey Errors-To: elfutils-devel-bounces+patchwork=sourceware.org@sourceware.org Sender: "Elfutils-devel" Since commit 4b42d9ad, libmicrohttpd's epoll event loop is used when available in which case we must disable its setting for spawning a thread per request. This contradicts the debuginfod doc's description of '-C', which indicates that if this command line option is not given then the thread pool size is unbounded. Fix this by using an auto-sized thread pool when '-C' is not given, just as we do when it's given with no argument. Update the doc's description of '-C'. Also use a fixed-size pool even if epoll is not supported. The unbounded pool config cannot be considered entirely reliable as it appears to cause random fails in the run-debuginfod-webapi-concurrency test. Signed-off-by: Aaron Merey --- debuginfod/ChangeLog | 7 +++++++ debuginfod/debuginfod.cxx | 22 ++++++++++------------ doc/ChangeLog | 4 ++++ doc/debuginfod.8 | 10 ++++------ 4 files changed, 25 insertions(+), 18 deletions(-) diff --git a/debuginfod/ChangeLog b/debuginfod/ChangeLog index da58e9a8..c692a389 100644 --- a/debuginfod/ChangeLog +++ b/debuginfod/ChangeLog @@ -1,3 +1,10 @@ +2022-09-02 Aaron Merey + + * debuginfod.cxx (parse_opt): If '-C' is given with no arg, do not + update connection_pool since it will be done at a later point. + (main): Use auto-sized connection_pool if '-C' isn't given with an + arg. Do not use MHD_USE_THREAD_PER_CONNECTION. + 2022-08-17 Martin Liska * debuginfod.cxx (handle_buildid): Update HTTP statistics only if diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx index 3e2dd9ef..8680c048 100644 --- a/debuginfod/debuginfod.cxx +++ b/debuginfod/debuginfod.cxx @@ -575,8 +575,6 @@ parse_opt (int key, char *arg, if (connection_pool < 2) argp_failure(state, 1, EINVAL, "-C NUM minimum 2"); } - else // arg not given - connection_pool = std::thread::hardware_concurrency() * 2 ?: 2; break; case 'I': // NB: no problem with unconditional free here - an earlier failed regcomp would exit program @@ -3937,6 +3935,11 @@ main (int argc, char *argv[]) } } + /* If '-C' wasn't given or was given with no arg, pick a reasonable default + for the number of worker threads. */ + if (connection_pool == 0) + connection_pool = std::thread::hardware_concurrency() * 2 ?: 2; + /* Note that MHD_USE_EPOLL and MHD_USE_THREAD_PER_CONNECTION don't work together. */ unsigned int use_epoll = 0; @@ -3944,12 +3947,11 @@ main (int argc, char *argv[]) use_epoll = MHD_USE_EPOLL; #endif - unsigned int mhd_flags = ((connection_pool || use_epoll - ? 0 : MHD_USE_THREAD_PER_CONNECTION) + unsigned int mhd_flags = ( #if MHD_VERSION >= 0x00095300 - | MHD_USE_INTERNAL_POLLING_THREAD + MHD_USE_INTERNAL_POLLING_THREAD #else - | MHD_USE_SELECT_INTERNALLY + MHD_USE_SELECT_INTERNALLY #endif | MHD_USE_DUAL_STACK | use_epoll @@ -3964,12 +3966,8 @@ main (int argc, char *argv[]) handler_cb, NULL, /* handler callback */ MHD_OPTION_EXTERNAL_LOGGER, error_cb, NULL, - (connection_pool - ? MHD_OPTION_THREAD_POOL_SIZE - : MHD_OPTION_END), - (connection_pool - ? (int)connection_pool - : MHD_OPTION_END), + MHD_OPTION_THREAD_POOL_SIZE, + (int)connection_pool, MHD_OPTION_END); MHD_Daemon *d4 = NULL; diff --git a/doc/ChangeLog b/doc/ChangeLog index ceec2467..c446e99e 100644 --- a/doc/ChangeLog +++ b/doc/ChangeLog @@ -1,3 +1,7 @@ +2022-09-02 Aaron Merey + + * debuginfod.8 (-C): Update description. + 2022-06-03 Michael Trapp * debuginfod.8 (--disable-source-scan): Document. diff --git a/doc/debuginfod.8 b/doc/debuginfod.8 index 50fce7f5..7c1dc3dd 100644 --- a/doc/debuginfod.8 +++ b/doc/debuginfod.8 @@ -212,15 +212,13 @@ following table summarizes the interpretaton of this option and its optional NUM parameter. .TS l l. -no option clone new thread for every request, no fixed pool -\-C use a fixed thread pool sized automatically +no option, \-C use a fixed thread pool sized automatically \-C=NUM use a fixed thread pool sized NUM, minimum 2 .TE -The first mode is useful for friendly bursty traffic. The second mode -is a simple and safe configuration based on the number of processors. -The third mode is suitable for tuned load-limiting configurations -facing unruly traffic. +The first mode is a simple and safe configuration based on the number +of processors. The second mode is suitable for tuned load-limiting +configurations facing unruly traffic. .TP .B "\-L"