From patchwork Tue Aug 30 14:43:05 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marius Bakke X-Patchwork-Id: 15069 Received: (qmail 13492 invoked by uid 89); 30 Aug 2016 14:43:42 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Checked: by ClamAV 0.99.2 on sourceware.org X-Virus-Found: No X-Spam-SWARE-Status: No, score=-3.3 required=5.0 tests=AWL, BAYES_00, FREEMAIL_FROM, RP_MATCHES_RCVD, SPF_PASS autolearn=ham version=3.3.2 spammy=julia, 5267, (unknown), mpc X-Spam-Status: No, score=-3.3 required=5.0 tests=AWL, BAYES_00, FREEMAIL_FROM, RP_MATCHES_RCVD, SPF_PASS autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on sourceware.org X-Spam-Level: X-HELO: lists.gnu.org Received: from lists.gnu.org (HELO lists.gnu.org) (208.118.235.17) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 30 Aug 2016 14:43:31 +0000 Received: from localhost ([::1]:49475 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bekGA-0004s4-90 for patchwork@sourceware.org; Tue, 30 Aug 2016 10:43:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49366) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bekG2-0004iw-Pq for guix-devel@gnu.org; Tue, 30 Aug 2016 10:43:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bekFy-0005HC-G5 for guix-devel@gnu.org; Tue, 30 Aug 2016 10:43:21 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:34568) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bekFv-0005Ep-Vj for guix-devel@gnu.org; Tue, 30 Aug 2016 10:43:18 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 7642F2028B; Tue, 30 Aug 2016 10:43:07 -0400 (EDT) Received: from frontend2 ([10.202.2.161]) by compute4.internal (MEProxy); Tue, 30 Aug 2016 10:43:07 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.com; h=cc :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=dbHDa N+LwrEJ9eMMe4Sbbm6Jp2s=; b=bF1Y9clYbQn8DG/J/TvVE46dScrsFJDjnCUW/ TxpxDlmzQ6u4OkWKR2d0UjnVd5XPvAz3WSfRAoFOY8iDKJ5ILHsnptdZK7OXpNX9 Wua/qpyppUASwl+VjXyTRo43zisj63FjMzTG3m6GTWaNROqFziWflCoILEmlqaIr OC/TsA= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-sasl-enc :x-sasl-enc; s=smtpout; bh=dbHDaN+LwrEJ9eMMe4Sbbm6Jp2s=; b=SEJrT TfMKuU8vaANPhW3B1xBue9EWTG9HTkuNYfx+SKXYzOZ8jBP7Ce2TxWWSKiP8Umop 8e/hwJ00Ak8kWIU4zSpAIJGRJ3AyxIRxaUViYeeGBsEeOtvjSnb2/o6xwVP0jJfF aGgz/XTsjtSzhk4MlkVeXKJt2jJzmWxZqK5zsU= X-Sasl-enc: 0jDYkKANffuVE3c/iDbw1i45Huoqhcs99R0iOdH1ALAf 1472568186 Received: from localhost (05430ba6.skybroadband.com [5.67.11.166]) by mail.messagingengine.com (Postfix) with ESMTPA id DEB0BCCE66; Tue, 30 Aug 2016 10:43:06 -0400 (EDT) From: Marius Bakke To: Leo Famulari Subject: Re: [PATCH] gnu: Add dlib. In-Reply-To: <878tvm2ddh.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> References: <20160816234507.GA24224@jasmine> <87h9ajzc1z.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> <20160818202353.GB2393@jasmine> <8737m1yqru.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> <20160821201757.GA21038@jasmine> <6004b842-2434-9cdc-7a31-04231eac5941@uq.edu.au> <87k2f957wt.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> <8760qr5o8n.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> <87eg5e4g4r.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> <20160824172608.GA24668@jasmine> <878tvm2ddh.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> Date: Tue, 30 Aug 2016 15:43:05 +0100 Message-ID: <87mvjuz5ae.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 66.111.4.28 X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel , Alex Kost Errors-To: guix-devel-bounces+patchwork=sourceware.org@gnu.org Sender: "Guix-devel" Marius Bakke writes: > Leo Famulari writes: > >> On Wed, Aug 24, 2016 at 11:26:28AM +0100, Marius Bakke wrote: >>> There are a couple of things going on in this thread: >>> >>> 1. Segfault on x86_64. This seems to have been resolved simply by >>> updating OpenBLAS. At least, I'm no longer able to reproduce it even >>> with LAPACK in inputs. So, that should fix the Hydra x86_64 build. >>> Can the OpenBLAS update be cherry-picked to master? >> >> I'd say it depends on whether the OpenBLAS users are building >> successfully on core-updates, but unfortunately core-updates is >> currently failing early in the bootstrap process [0]. Can you take a >> look at `guix refresh -l dlib` and pick some important looking >> applications to test with the updated OpenBLAS? > > I'm currently building the following openblas dependents: `libreoffice > bamm python-biom-format clipper shogun armadillo julia` and will try to > test BLAS functionality in some of them. Shogun failed to build in this run. I don't have time to investigate further, so picking the OpenBLAS update is not very appealing. Instead I opted to disable the test that fails with lapack (and without, on Hydra), since it's one specific openblas operation that is not unique to dlib. I think it's an acceptable tradeoff, to give users the full dlib functionality, and have the segfault "sort itself" when core-updates lands in master. >>> 2. i686 test failures. Updating OpenBLAS fixed 1/5 errors. The remaining >>> four are reproducible on 32-bit Ubuntu, so they do not seem Guix >>> related. Upstream has been notified. >>> >>> 3. ARM failures. I don't have ARM hardware to test on, but I'm guessing >>> it's similar to i686 (i.e. not directly Guix related). >> >> Maybe dlib is 64-bit only? If that's the case, we can disable it on >> those architectures. > > According to the developer[0], these targets should be supported. > > 0: https://github.com/davisking/dlib/issues/197 > > We could disable tests (at least the failing ones) on these platforms > until this issue is resolved. The mips64el target on Hydra times out > after 3600 seconds on one of the tests, but seems fine up to that point. > Some of these tests are fairly CPU heavy, so the timeout may be too low. Below is a patch which disables these tests (and the above segfault) for 19.1, rather than backporting the patches from dlib master branch. One note about the patch: I could not figure out how to pass the list of tests as arguments to `substitute*`, so currently it calls `substitute*` for each of them. Any tips to prevent this? It also no longer builds the main application twice for tests. From 121895b2d7915b1f1ef0430921418783019393d0 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sat, 27 Aug 2016 17:23:58 +0100 Subject: [PATCH 2/2] gnu: dlib: Disable failing tests. * gnu/packages/machine-learning.scm (dlib)[arguments]: Add 'disable-failing-tests phase and prevent building dlib twice. [inputs]: Add lapack. --- gnu/packages/machine-learning.scm | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 7669702..35f1514 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -499,14 +499,25 @@ single hidden layer, and for multinomial log-linear models.") (substitute* "dlib/config.h" (("^//#define DLIB_DISABLE_ASSERTS") "#define DLIB_DISABLE_ASSERTS")) #t)) + (add-after 'disable-asserts 'disable-failing-tests + ;; A number of tests are known to fail on 32-bit platforms in 19.1. + ;; See https://github.com/davisking/dlib/issues/197 for details. + (lambda _ + (for-each + (lambda (test) + (substitute* "dlib/test/makefile" + (((string-append "SRC \\+= " test "\\.cpp")) "")) #t) + (list "learning_to_track" "max_cost_assignment" ; armhf + "optimization" "matrix2" "mpc" ; i686 + "empirical_map" ; may segfault with < openblas-0.2.18 + "object_detector")))) ; timeout on mips64el (replace 'check (lambda _ ;; No test target, so we build and run the unit tests here. - (let ((test-dir (string-append "../dlib-" ,version "/dlib/test/build"))) - (mkdir-p test-dir) + (let ((test-dir (string-append "../dlib-" ,version "/dlib/test"))) (with-directory-excursion test-dir - (and (zero? (system* "cmake" "..")) - (zero? (system* "cmake" "--build" "." "--config" "Release")) + (setenv "CXXFLAGS" "-std=gnu++11") + (and (zero? (system* "make" "-j" (number->string (parallel-job-count)))) (zero? (system* "./dtest" "--runall"))))))) (add-after 'install 'delete-static-library (lambda* (#:key outputs #:allow-other-keys) @@ -515,7 +526,7 @@ single hidden layer, and for multinomial log-linear models.") `(("pkg-config" ,pkg-config))) (inputs `(("giflib" ,giflib) - ;("lapack" ,lapack) XXX lapack here causes test failures in some setups. + ("lapack" ,lapack) ("libjpeg" ,libjpeg) ("libpng" ,libpng) ("libx11" ,libx11) -- 2.9.3