diff mbox

Python 3.5 start of update

Message ID 87mvmetukz.fsf@dustycloud.org
State New
Headers show

Commit Message

Christopher Allan Webber June 21, 2016, 3:44 p.m. UTC
Building on both the work that Diane did and my recent patch for python
3.4.4, here's two patches which do both consecutively: the first patch
updates python-3.4 to 3.4.4, and the second patch adds python-3.5 and
sets this to be the default python.

I'm interested in packaging Pulsar, but this requires Python 3.5 or
higher, so these patches are a step towards it.

Most of the work is Diane's.  Thank you Diane!  Great work :)

 - Chris

Comments

Ludovic Courtès June 21, 2016, 9:29 p.m. UTC | #1
Christopher Allan Webber <cwebber@dustycloud.org> skribis:

> Building on both the work that Diane did and my recent patch for python
> 3.4.4, here's two patches which do both consecutively: the first patch
> updates python-3.4 to 3.4.4, and the second patch adds python-3.5 and
> sets this to be the default python.
>
> I'm interested in packaging Pulsar, but this requires Python 3.5 or
> higher, so these patches are a step towards it.
>
> Most of the work is Diane's.  Thank you Diane!  Great work :)

Nice!

> From bbc1f68204318d79ece3fa46ed71d7c21d6ec8a2 Mon Sep 17 00:00:00 2001
> From: Christopher Allan Webber <cwebber@dustycloud.org>
> Date: Fri, 29 Apr 2016 17:53:50 -0500
> Subject: [PATCH] gnu: Add geiser-next.

Oops.  ;-)

> From ec0797ceaed8f20b4dbbfb8fb96b2c4752e93f2e Mon Sep 17 00:00:00 2001
> From: Diane Trout <diane@ghic.org>
> Date: Tue, 21 Jun 2016 09:22:43 -0500
> Subject: [PATCH 2/2] gnu: Add python-3.5.
>
> * gnu/packages/python.scm (python-3.5): New variable.
>   (python-3.4): Inherit from python-3.5.
>   (python-3): Switched alias to point to python-3.5.
>   (python-minimal): Use the system-ffi as without-system-ffi caused the build
>   to fail when trying to import _ctypes.
> * gnu/packages/patches/python-fix-tests-py3.5.patch:
>   New file.  Patch Lib/test/test_pathlib.py to skip test_expanduser.
>   Patch Lib/test/test_tarfile.py to ignore pwd.getpwuid and grp.getgrgid tests
>   for user root being of id 0.

Feel free to add a “Co-authored-by” tag if you think it’s relevant.

> diff --git a/gnu/packages/patches/python-fix-tests-py3.5.patch b/gnu/packages/patches/python-fix-tests-py3.5.patch
> new file mode 100644
> index 0000000..47dded4
> --- /dev/null
> +++ b/gnu/packages/patches/python-fix-tests-py3.5.patch
> @@ -0,0 +1,35 @@
> +Additional test fixes which affect Python 3.5 (and presumably later) but not
> +prior revisions of Python.
> +
> +--- Lib/test/test_pathlib.py     2014-03-01 03:02:36.088311000 +0100
> ++++ Lib/test/test_pathlib.py     2014-03-01 04:56:37.768311000 +0100
> +@@ -1986,8 +1986,9 @@
> +         expect = set() if not support.fs_is_case_insensitive(BASE) else given
> +         self.assertEqual(given, expect)
> +         self.assertEqual(set(p.rglob("FILEd*")), set())
> + 
> ++    @unittest.skipIf(True, "Guix builder home is '/' which causes trouble for these tests")

Could it be that adding a phase that does (setenv "HOME" (getcwd)) would
help?

> +--- Lib/test/test_tarfile.py        2016-02-24 19:22:52.597208055 +0000
> ++++ Lib/test/test_tarfile.py     2016-02-24 20:50:48.941950135 +0000
> +@@ -2305,11 +2305,14 @@
> +     try:
> +         import pwd, grp
> +     except ImportError:
> +         return False
> +-    if pwd.getpwuid(0)[0] != 'root':
> +-        return False
> +-    if grp.getgrgid(0)[0] != 'root':
> ++    try:
> ++        if pwd.getpwuid(0)[0] != 'root':
> ++            return False
> ++        if grp.getgrgid(0)[0] != 'root':
> ++            return False
> ++    except KeyError:
> +         return False
> +     return True

(I was skeptical about this bit so I tried this:

--8<---------------cut here---------------start------------->8---
store-monad@(guile-user) [1]> (gexp->derivation "foo"
						#~(call-with-output-file #$output
						    (lambda (port)
						      (write (list (false-if-exception (getpwuid 0))
								   (false-if-exception (getgrgid 0)))
							     port))))
$6 = #<derivation /gnu/store/zxlirg8wbl8r5pkkhsc89gxvx6lzmwd7-foo.drv => /gnu/store/fr9klqdkfqz8ayiq1fjq5i468qjx39mc-foo 42843c0>
store-monad@(guile-user) [1]> (built-derivations (list $6))
substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
building path(s) `/gnu/store/fr9klqdkfqz8ayiq1fjq5i468qjx39mc-foo'
$7 = #t
--8<---------------cut here---------------end--------------->8---

… and indeed, we get (#f #f).  No wonder because libstore/build.cc
does not add any entry for “root” in /etc/passwd.  Probably worth fixing
someday.)

Other than that, LGTM!

The big frustration is that we’ll have to open a ‘python-updates’
branch, where we can also add Hartmut’s changes, but we’ll have to wait
until ‘core-updates’ is merged before we can start building it, which
could take another week or so.

Thank you!

Ludo’.
Christopher Allan Webber June 22, 2016, 1:15 p.m. UTC | #2
Ludovic Courtès writes:

>> From bbc1f68204318d79ece3fa46ed71d7c21d6ec8a2 Mon Sep 17 00:00:00 2001
>> From: Christopher Allan Webber <cwebber@dustycloud.org>
>> Date: Fri, 29 Apr 2016 17:53:50 -0500
>> Subject: [PATCH] gnu: Add geiser-next.
>
> Oops.  ;-)

Oops indeed... I'll attach the right first of 2 patches when I get the
next commit in.

>> From ec0797ceaed8f20b4dbbfb8fb96b2c4752e93f2e Mon Sep 17 00:00:00 2001
>> From: Diane Trout <diane@ghic.org>
>> Date: Tue, 21 Jun 2016 09:22:43 -0500
>> Subject: [PATCH 2/2] gnu: Add python-3.5.
>>
>> * gnu/packages/python.scm (python-3.5): New variable.
>>   (python-3.4): Inherit from python-3.5.
>>   (python-3): Switched alias to point to python-3.5.
>>   (python-minimal): Use the system-ffi as without-system-ffi caused the build
>>   to fail when trying to import _ctypes.
>> * gnu/packages/patches/python-fix-tests-py3.5.patch:
>>   New file.  Patch Lib/test/test_pathlib.py to skip test_expanduser.
>>   Patch Lib/test/test_tarfile.py to ignore pwd.getpwuid and grp.getgrgid tests
>>   for user root being of id 0.
>
> Feel free to add a “Co-authored-by” tag if you think it’s relevant.

Oh I didn't know about that... will do.

>> diff --git a/gnu/packages/patches/python-fix-tests-py3.5.patch b/gnu/packages/patches/python-fix-tests-py3.5.patch
>> new file mode 100644
>> index 0000000..47dded4
>> --- /dev/null
>> +++ b/gnu/packages/patches/python-fix-tests-py3.5.patch
>> @@ -0,0 +1,35 @@
>> +Additional test fixes which affect Python 3.5 (and presumably later) but not
>> +prior revisions of Python.
>> +
>> +--- Lib/test/test_pathlib.py     2014-03-01 03:02:36.088311000 +0100
>> ++++ Lib/test/test_pathlib.py     2014-03-01 04:56:37.768311000 +0100
>> +@@ -1986,8 +1986,9 @@
>> +         expect = set() if not support.fs_is_case_insensitive(BASE) else given
>> +         self.assertEqual(given, expect)
>> +         self.assertEqual(set(p.rglob("FILEd*")), set())
>> + 
>> ++    @unittest.skipIf(True, "Guix builder home is '/' which causes trouble for these tests")
>
> Could it be that adding a phase that does (setenv "HOME" (getcwd)) would
> help?

A good idea, will give it a try!

>> +--- Lib/test/test_tarfile.py        2016-02-24 19:22:52.597208055 +0000
>> ++++ Lib/test/test_tarfile.py     2016-02-24 20:50:48.941950135 +0000
>> +@@ -2305,11 +2305,14 @@
>> +     try:
>> +         import pwd, grp
>> +     except ImportError:
>> +         return False
>> +-    if pwd.getpwuid(0)[0] != 'root':
>> +-        return False
>> +-    if grp.getgrgid(0)[0] != 'root':
>> ++    try:
>> ++        if pwd.getpwuid(0)[0] != 'root':
>> ++            return False
>> ++        if grp.getgrgid(0)[0] != 'root':
>> ++            return False
>> ++    except KeyError:
>> +         return False
>> +     return True
>
> (I was skeptical about this bit so I tried this:
>
> --8<---------------cut here---------------start------------->8---
> store-monad@(guile-user) [1]> (gexp->derivation "foo"
> 						#~(call-with-output-file #$output
> 						    (lambda (port)
> 						      (write (list (false-if-exception (getpwuid 0))
> 								   (false-if-exception (getgrgid 0)))
> 							     port))))
> $6 = #<derivation /gnu/store/zxlirg8wbl8r5pkkhsc89gxvx6lzmwd7-foo.drv => /gnu/store/fr9klqdkfqz8ayiq1fjq5i468qjx39mc-foo 42843c0>
> store-monad@(guile-user) [1]> (built-derivations (list $6))
> substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
> building path(s) `/gnu/store/fr9klqdkfqz8ayiq1fjq5i468qjx39mc-foo'
> $7 = #t
> --8<---------------cut here---------------end--------------->8---
>
> … and indeed, we get (#f #f).  No wonder because libstore/build.cc
> does not add any entry for “root” in /etc/passwd.  Probably worth fixing
> someday.)

Heh :)

Is it worth filing a bug?

> Other than that, LGTM!
>
> The big frustration is that we’ll have to open a ‘python-updates’
> branch, where we can also add Hartmut’s changes, but we’ll have to wait
> until ‘core-updates’ is merged before we can start building it, which
> could take another week or so.

Makes sense.  It'll be worth it!

I'll try to get the rest of this done this afternoon, but it might
happen sometime later this week.

 - Chris
diff mbox

Patch

From ec0797ceaed8f20b4dbbfb8fb96b2c4752e93f2e Mon Sep 17 00:00:00 2001
From: Diane Trout <diane@ghic.org>
Date: Tue, 21 Jun 2016 09:22:43 -0500
Subject: [PATCH 2/2] gnu: Add python-3.5.

* gnu/packages/python.scm (python-3.5): New variable.
  (python-3.4): Inherit from python-3.5.
  (python-3): Switched alias to point to python-3.5.
  (python-minimal): Use the system-ffi as without-system-ffi caused the build
  to fail when trying to import _ctypes.
* gnu/packages/patches/python-fix-tests-py3.5.patch:
  New file.  Patch Lib/test/test_pathlib.py to skip test_expanduser.
  Patch Lib/test/test_tarfile.py to ignore pwd.getpwuid and grp.getgrgid tests
  for user root being of id 0.
---
 gnu/packages/patches/python-fix-tests-py3.5.patch | 35 +++++++++++++++++++++++
 gnu/packages/python.scm                           | 31 ++++++++++++++------
 2 files changed, 58 insertions(+), 8 deletions(-)
 create mode 100644 gnu/packages/patches/python-fix-tests-py3.5.patch

diff --git a/gnu/packages/patches/python-fix-tests-py3.5.patch b/gnu/packages/patches/python-fix-tests-py3.5.patch
new file mode 100644
index 0000000..47dded4
--- /dev/null
+++ b/gnu/packages/patches/python-fix-tests-py3.5.patch
@@ -0,0 +1,35 @@ 
+Additional test fixes which affect Python 3.5 (and presumably later) but not
+prior revisions of Python.
+
+--- Lib/test/test_pathlib.py     2014-03-01 03:02:36.088311000 +0100
++++ Lib/test/test_pathlib.py     2014-03-01 04:56:37.768311000 +0100
+@@ -1986,8 +1986,9 @@
+         expect = set() if not support.fs_is_case_insensitive(BASE) else given
+         self.assertEqual(given, expect)
+         self.assertEqual(set(p.rglob("FILEd*")), set())
+ 
++    @unittest.skipIf(True, "Guix builder home is '/' which causes trouble for these tests")
+     def test_expanduser(self):
+         P = self.cls
+         support.import_module('pwd')
+         import pwd
+--- Lib/test/test_tarfile.py        2016-02-24 19:22:52.597208055 +0000
++++ Lib/test/test_tarfile.py     2016-02-24 20:50:48.941950135 +0000
+@@ -2305,11 +2305,14 @@
+     try:
+         import pwd, grp
+     except ImportError:
+         return False
+-    if pwd.getpwuid(0)[0] != 'root':
+-        return False
+-    if grp.getgrgid(0)[0] != 'root':
++    try:
++        if pwd.getpwuid(0)[0] != 'root':
++            return False
++        if grp.getgrgid(0)[0] != 'root':
++            return False
++    except KeyError:
+         return False
+     return True
+ 
+     
\ No newline at end of file
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 92535cb..0cb08ed 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -20,6 +20,7 @@ 
 ;;; Copyright © 2016 Danny Milosavljevic <dannym+a@scratchpost.org>
 ;;; Copyright © 2016 Lukas Gradl <lgradl@openmailbox.org>
 ;;; Copyright © 2016 Hartmut Goebel <h.goebel@crazy-compilers.com>
+;;; Copyright © 2016 Diane Trout <diane@ghic.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -269,21 +270,22 @@  data types.")
 ;; Current 2.x version.
 (define-public python-2 python-2.7)
 
-(define-public python-3.4
+(define-public python-3.5
   (package (inherit python-2)
-    (version "3.4.4")
+    (version "3.5.1")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://www.python.org/ftp/python/"
                                   version "/Python-" version ".tar.xz"))
               (patches (search-patches
                         "python-fix-tests.patch"
+                        "python-fix-tests-py3.5.patch"
                         "python-3-deterministic-build-info.patch"
                         "python-3-search-paths.patch"))
               (patch-flags '("-p0"))
               (sha256
                (base32
-                "18kb5c29w04rj4gyz3jngm72sy8izfnbjlm6ajv6rv2m061d75x7"))))
+                "1j95yx32ggqx8jf13h3c8qfp34ixpyg8ipqcdjmn143d6q67rmf6"))))
     (arguments (substitute-keyword-arguments (package-arguments python-2)
                  ((#:tests? _) #t)))
     (native-search-paths
@@ -293,8 +295,24 @@  data types.")
                                         (version-major+minor version)
                                         "/site-packages"))))))))
 
+(define-public python-3.4
+  (package (inherit python-3.5)
+    (version "3.4.4")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://www.python.org/ftp/python/"
+                                  version "/Python-" version ".tar.xz"))
+              (patches (search-patches
+                        "python-fix-tests.patch"
+                        "python-3-deterministic-build-info.patch"
+                        "python-3-search-paths.patch"))
+              (patch-flags '("-p0"))
+              (sha256
+               (base32
+                "18kb5c29w04rj4gyz3jngm72sy8izfnbjlm6ajv6rv2m061d75x7"))))))
+
 ;; Current 3.x version.
-(define-public python-3 python-3.4)
+(define-public python-3 python-3.5)
 
 ;; Current major version.
 (define-public python python-3)
@@ -316,14 +334,11 @@  data types.")
   (package (inherit python)
     (name "python-minimal")
     (outputs '("out"))
-    (arguments
-     (substitute-keyword-arguments (package-arguments python)
-       ((#:configure-flags cf)
-        `(append ,cf '("--without-system-ffi")))))
 
     ;; OpenSSL is a mandatory dependency of Python 3.x, for urllib;
     ;; zlib is required by 'zipimport', used by pip.
     (inputs `(("openssl" ,openssl)
+              ("libffi" ,libffi)                         ; for ctypes
               ("zlib" ,zlib)))))
 
 (define* (wrap-python3 python
-- 
2.8.4