diff mbox

[2/3] gnu: Add python-pyxb.

Message ID 8737kqzk30.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me
State New
Headers show

Commit Message

Marius Bakke Sept. 23, 2016, 3:56 p.m. UTC
Ben Woodcroft <b.woodcroft@uq.edu.au> writes:

>>
>>> Also, I notice that pysam bundles htslib, bcftools and samtools C code.
>>> Hopefully it should be straightforward enough to remove htslib as there
>>> are install instructions, I'm not sure about the other two. This
>>> shouldn't block the patch here, but would you mind taking a look?
>>> http://pysam.readthedocs.io/en/latest/installation.html#installation
>> I had a go at this, and also enabled tests since I was reading the build
>> system anyway. Samtools and bcftools does not seem possible to un-bundle
>> at this time, but htslib was straightforward.
> OK. I don't think it needs to be propagated though, right? Also, would 
> you mind separating the change to modify-phases syntax and unbundling of 
> htslib into two patches please? Other than that this whole series LGTM.

Modify-phases is now a separate patch. Htslib is propagated because it
is included by the installed samtools and bcftools headers; at least one
of pysams dependencies failed to build when it was a regular input.

I added some comments and other cosmetic changes. Updated patch below.

Cheers!

Comments

Ben Woodcroft Sept. 24, 2016, 12:23 a.m. UTC | #1
On 24/09/16 01:56, Marius Bakke wrote:
> Ben Woodcroft <b.woodcroft@uq.edu.au> writes:
>
>>>> Also, I notice that pysam bundles htslib, bcftools and samtools C code.
>>>> Hopefully it should be straightforward enough to remove htslib as there
>>>> are install instructions, I'm not sure about the other two. This
>>>> shouldn't block the patch here, but would you mind taking a look?
>>>> http://pysam.readthedocs.io/en/latest/installation.html#installation
>>> I had a go at this, and also enabled tests since I was reading the build
>>> system anyway. Samtools and bcftools does not seem possible to un-bundle
>>> at this time, but htslib was straightforward.
>> OK. I don't think it needs to be propagated though, right? Also, would
>> you mind separating the change to modify-phases syntax and unbundling of
>> htslib into two patches please? Other than that this whole series LGTM.
> Modify-phases is now a separate patch.
Thanks.

> Htslib is propagated because it
> is included by the installed samtools and bcftools headers; at least one
> of pysams dependencies failed to build when it was a regular input.
Ah, I see. Can this be avoided with patchelf?
ben
Marius Bakke Sept. 24, 2016, 3:15 a.m. UTC | #2
Ben Woodcroft <b.woodcroft@uq.edu.au> writes:

>> Htslib is propagated because it
>> is included by the installed samtools and bcftools headers; at least one
>> of pysams dependencies failed to build when it was a regular input.
> Ah, I see. Can this be avoided with patchelf?

I don't think so. The htslib headers need to be propagated for anything
that uses header files from pysam, since they contain contain code such
as "include <htslib/foo.h>". Perhaps you can update the comment to make
it more clear?

Patchelf is probably what made that work when htslib was bundled.

Cheers,
Marius
diff mbox

Patch

From eaa38fb4174997f6282ff7d278a6aa4153b75bcf Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke@fastmail.com>
Date: Fri, 16 Sep 2016 18:05:15 +0100
Subject: [PATCH 1/7] gnu: python-pysam: Update to 0.9.1.4.

* gnu/packages/bioinformatics.scm (python-pysam, python2-pysam): Update
to 0.9.1.4.
---
 gnu/packages/bioinformatics.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 1bf91a9..6e83461 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -1290,13 +1290,13 @@  multiple sequence alignments.")
 (define-public python-pysam
   (package
     (name "python-pysam")
-    (version "0.8.4")
+    (version "0.9.1.4")
     (source (origin
               (method url-fetch)
               (uri (pypi-uri "pysam" version))
               (sha256
                (base32
-                "1slx5mb94mzm5qzk52q270sab0sar95j67w1g1k452nz3s9j7krh"))))
+                "1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn"))))
     (build-system python-build-system)
     (arguments
      `(#:tests? #f ; tests are excluded in the manifest
-- 
2.10.0


From 5431b05fe66398f5c44a1f0a81e1e90663550ac1 Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke@fastmail.com>
Date: Sat, 17 Sep 2016 10:33:28 +0100
Subject: [PATCH 2/7] gnu: Add python-pyxb.

* gnu/packages/xml.scm (python-pyxb, python2-pyxb): New variables.
---
 gnu/packages/xml.scm | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/gnu/packages/xml.scm b/gnu/packages/xml.scm
index 7befad5..cd3426b 100644
--- a/gnu/packages/xml.scm
+++ b/gnu/packages/xml.scm
@@ -13,6 +13,7 @@ 
 ;;; Copyright © 2016 Jan Nieuwenhuizen <janneke@gnu.org>
 ;;; Copyright © 2016 ng0 <ng0@we.make.ritual.n0.is>
 ;;; Copyright © 2016 Tobias Geerinckx-Rice <me@tobias.gr>
+;;; Copyright © 2016 Marius Bakke <mbakke@fastmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -729,6 +730,30 @@  different Unicode encodings which happen automatically during
 parsing/saving.")
     (license license:expat)))
 
+(define-public python-pyxb
+  (package
+    (name "python-pyxb")
+    (version "1.2.5")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "PyXB" version))
+              (sha256
+               (base32
+                "0rzzwibfqa28gxgcxx4cybx1qcg0g6fand06ykj3gz7z5kp653sf"))))
+    (build-system python-build-system)
+    (home-page "http://pyxb.sourceforge.net/")
+    (synopsis "Python XML Schema Bindings")
+    (description
+     "PyXB (\"pixbee\") is a pure Python package that generates Python source
+code for classes that correspond to data structures defined by XMLSchema.")
+    (license (list license:asl2.0    ; Most files.
+                   license:expat     ; pyxb/utils/six.py
+                   license:gpl2      ; bundled jquery in doc is dual MIT/GPL2
+                   license:psfl))))  ; pyxb/utils/activestate.py
+
+(define-public python2-pyxb
+  (package-with-python2 python-pyxb))
+
 (define-public xmlto
   (package
     (name "xmlto")
-- 
2.10.0


From 923fc43d8c665d1a2795ea2c221003e54544a06b Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke@fastmail.com>
Date: Sat, 17 Sep 2016 10:42:56 +0100
Subject: [PATCH 3/7] gnu: python2-pbcore: Update to 1.2.10.

* gnu/packages/bioinformatics.scm (python2-pbcore): Update to 1.2.10.
[propagated-inputs]: New field. Add python2-pyxb.
---
 gnu/packages/bioinformatics.scm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 6e83461..a935d5a 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -3376,13 +3376,13 @@  interrupted by stop codons.  OrfM finds and prints these ORFs.")
 (define-public python2-pbcore
   (package
     (name "python2-pbcore")
-    (version "1.2.8")
+    (version "1.2.10")
     (source (origin
               (method url-fetch)
               (uri (pypi-uri "pbcore" version))
               (sha256
                (base32
-                "02pfn5raa3zf739672bg0dkx7z3j2c4nx7vmpfjqy5b12jrqpymk"))))
+                "1kjmv891d6qbpp4shhhvkl02ff4q5xlpnls2513sm2cjcrs52f1i"))))
     (build-system python-build-system)
     (arguments `(#:python ,python-2)) ; pbcore requires Python 2.7
     (inputs
@@ -3395,6 +3395,8 @@  interrupted by stop codons.  OrfM finds and prints these ORFs.")
        ("python-nose" ,python2-nose)
        ("python-setuptools" ,python2-setuptools)
        ("python-sphinx" ,python2-sphinx)))
+    (propagated-inputs
+     `(("python-pyxb" ,python2-pyxb)))
     (home-page "http://pacificbiosciences.github.io/pbcore/")
     (synopsis "Library for reading and writing PacBio data files")
     (description
-- 
2.10.0


From 7748677c13242b7edb6ddae1efed7f0699e487bf Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke@fastmail.com>
Date: Thu, 22 Sep 2016 13:47:53 +0100
Subject: [PATCH 4/7] gnu: Add bcftools.

* gnu/packages/bioinformatics.scm (bcftools): New variable.
---
 gnu/packages/bioinformatics.scm | 58 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index a935d5a..679502e 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -265,6 +265,64 @@  instance, it implements several methods to assess contig-wise read coverage.")
 BAM files.")
     (license license:expat)))
 
+(define-public bcftools
+  (package
+    (name "bcftools")
+    (version "1.3.1")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append
+                    "https://github.com/samtools/bcftools/releases/download/"
+                    version "/bcftools-" version ".tar.bz2"))
+              (sha256
+               (base32
+                "095ry68vmz9q5s1scjsa698dhgyvgw5aicz24c19iwfbai07mhqj"))
+              (modules '((guix build utils)))
+              (snippet
+               ;; Delete bundled htslib.
+               '(delete-file-recursively "htslib-1.3.1"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:test-target "test"
+       #:make-flags
+       (list
+        "USE_GPL=1"
+        (string-append "prefix=" (assoc-ref %outputs "out"))
+        (string-append "HTSDIR=" (assoc-ref %build-inputs "htslib") "/include")
+        (string-append "HTSLIB=" (assoc-ref %build-inputs "htslib") "/lib/libhts.a")
+        (string-append "BGZIP=" (assoc-ref %build-inputs "htslib") "/bin/bgzip")
+        (string-append "TABIX=" (assoc-ref %build-inputs "htslib") "/bin/tabix"))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'patch-Makefile
+           (lambda _
+             (substitute* "Makefile"
+               ;; Do not attempt to build htslib.
+               (("^include \\$\\(HTSDIR\\)/htslib\\.mk") "")
+               ;; Link against GSL cblas.
+               (("-lcblas") "-lgslcblas"))
+             #t))
+         (delete 'configure)
+         (add-before 'check 'patch-tests
+           (lambda _
+             (substitute* "test/test.pl"
+               (("/bin/bash") (which "bash")))
+             #t)))))
+    (native-inputs
+     `(("htslib" ,htslib)
+       ("perl" ,perl)))
+    (inputs
+     `(("gsl" ,gsl)
+       ("zlib" ,zlib)))
+    (home-page "https://samtools.github.io/bcftools/")
+    (synopsis "Utilities for variant calling and manipulating VCFs and BCFs")
+    (description
+     "BCFtools is a set of utilities that manipulate variant calls in the
+Variant Call Format (VCF) and its binary counterpart BCF.  All commands work
+transparently with both VCFs and BCFs, both uncompressed and BGZF-compressed.")
+    ;; The sources are dual MIT/GPL, but becomes GPL-only when USE_GPL=1.
+    (license (list license:gpl3+ license:expat))))
+
 (define-public bedops
   (package
     (name "bedops")
-- 
2.10.0


From d4fd6f8482a76ccfe4d33f80070445d284006166 Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke@fastmail.com>
Date: Fri, 23 Sep 2016 06:40:40 +0100
Subject: [PATCH 5/7] gnu: python-pysam: Use 'modify-phases'.

* gnu/packages/bioinformatics.scm (python-pysam)[arguments]: Use
'modify-phases'.
---
 gnu/packages/bioinformatics.scm | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 679502e..518dfea 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -1359,12 +1359,12 @@  multiple sequence alignments.")
     (arguments
      `(#:tests? #f ; tests are excluded in the manifest
        #:phases
-       (alist-cons-before
-        'build 'set-flags
-        (lambda _
-          (setenv "LDFLAGS" "-lncurses")
-          (setenv "CFLAGS" "-D_CURSES_LIB=1"))
-        %standard-phases)))
+       (modify-phases %standard-phases
+         (add-before 'build 'set-flags
+           (lambda _
+             (setenv "LDFLAGS" "-lncurses")
+             (setenv "CFLAGS" "-D_CURSES_LIB=1")
+             #t)))))
     (inputs
      `(("ncurses"           ,ncurses)
        ("zlib"              ,zlib)))
-- 
2.10.0


From e4d3befdc33a77f6b3d1d241befd7077c0be7e70 Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke@fastmail.com>
Date: Thu, 22 Sep 2016 14:08:05 +0100
Subject: [PATCH 6/7] gnu: python-pysam: Delete bundled htslib.

* gnu/packages/bioinformatics.scm (python-pysam)[source]: Add snippet
to delete htslib.
[arguments]: Add htslib flags in 'set-flags phase.
[propagated-inputs]: New field. Add htslib.
---
 gnu/packages/bioinformatics.scm | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 518dfea..a712a01 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -1354,17 +1354,28 @@  multiple sequence alignments.")
               (uri (pypi-uri "pysam" version))
               (sha256
                (base32
-                "1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn"))))
+                "1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn"))
+              (modules '((guix build utils)))
+              (snippet
+               ;; Drop bundled htslib. TODO: Also remove samtools and bcftools.
+               '(delete-file-recursively "htslib"))))
     (build-system python-build-system)
     (arguments
      `(#:tests? #f ; tests are excluded in the manifest
        #:phases
        (modify-phases %standard-phases
          (add-before 'build 'set-flags
-           (lambda _
+           (lambda* (#:key inputs #:allow-other-keys)
+             (setenv "HTSLIB_MODE" "external")
+             (setenv "HTSLIB_LIBRARY_DIR"
+                     (string-append (assoc-ref inputs "htslib") "/lib"))
+             (setenv "HTSLIB_INCLUDE_DIR"
+                     (string-append (assoc-ref inputs "htslib") "/include"))
              (setenv "LDFLAGS" "-lncurses")
              (setenv "CFLAGS" "-D_CURSES_LIB=1")
              #t)))))
+    (propagated-inputs
+     `(("htslib"            ,htslib))) ; Included from installed header files.
     (inputs
      `(("ncurses"           ,ncurses)
        ("zlib"              ,zlib)))
-- 
2.10.0


From 2afa68286db8cf55cc9dc325bef2f529b7e8f2ba Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke@fastmail.com>
Date: Thu, 22 Sep 2016 14:11:59 +0100
Subject: [PATCH 7/7] gnu: python-pysam: Enable tests.

* gnu/packages/bioinformatics.scm (python-pysam)[source]: Change from
PyPi to source archive due to missing test data.
[arguments]: Add check phase after install.
[native-inputs]: Add python-nose, samtools and bcftools.
---
 gnu/packages/bioinformatics.scm | 36 ++++++++++++++++++++++++++++++------
 1 file changed, 30 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index a712a01..7a784c8 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -1351,18 +1351,21 @@  multiple sequence alignments.")
     (version "0.9.1.4")
     (source (origin
               (method url-fetch)
-              (uri (pypi-uri "pysam" version))
+              ;; Test data is missing on PyPi.
+              (uri (string-append
+                    "https://github.com/pysam-developers/pysam/archive/v"
+                    version ".tar.gz"))
+              (file-name (string-append name "-" version ".tar.gz"))
               (sha256
                (base32
-                "1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn"))
+                "0y41ssbg6nvn2jgcbnrvkzblpjcwszaiv1rgyd8dwzjkrbfsgsmc"))
               (modules '((guix build utils)))
               (snippet
                ;; Drop bundled htslib. TODO: Also remove samtools and bcftools.
                '(delete-file-recursively "htslib"))))
     (build-system python-build-system)
     (arguments
-     `(#:tests? #f ; tests are excluded in the manifest
-       #:phases
+     `(#:phases
        (modify-phases %standard-phases
          (add-before 'build 'set-flags
            (lambda* (#:key inputs #:allow-other-keys)
@@ -1373,7 +1376,24 @@  multiple sequence alignments.")
                      (string-append (assoc-ref inputs "htslib") "/include"))
              (setenv "LDFLAGS" "-lncurses")
              (setenv "CFLAGS" "-D_CURSES_LIB=1")
-             #t)))))
+             #t))
+         (delete 'check)
+         (add-after 'install 'check
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (setenv "PYTHONPATH"
+                     (string-append
+                      (getenv "PYTHONPATH")
+                      ":" (assoc-ref outputs "out")
+                      "/lib/python"
+                      (string-take (string-take-right
+                                    (assoc-ref inputs "python") 5) 3)
+                      "/site-packages"))
+             ;; Step out of source dir so python does not import from CWD.
+             (chdir "tests")
+             (setenv "HOME" "/tmp")
+             (and (zero? (system* "make" "-C" "pysam_data"))
+                  (zero? (system* "make" "-C" "cbcf_data"))
+                  (zero? (system* "nosetests" "-v"))))))))
     (propagated-inputs
      `(("htslib"            ,htslib))) ; Included from installed header files.
     (inputs
@@ -1381,7 +1401,11 @@  multiple sequence alignments.")
        ("zlib"              ,zlib)))
     (native-inputs
      `(("python-cython"     ,python-cython)
-       ("python-setuptools" ,python-setuptools)))
+       ("python-setuptools" ,python-setuptools)
+       ;; Depencies below are are for tests only.
+       ("samtools"          ,samtools)
+       ("bcftools"          ,bcftools)
+       ("python-nose"       ,python-nose)))
     (home-page "https://github.com/pysam-developers/pysam")
     (synopsis "Python bindings to the SAMtools C API")
     (description
-- 
2.10.0