[RFC] Make libarchive python module optional. Was Re: [Bug default/31672] abidb: make libarchive a soft dependency

  Hello,

"fche at redhat dot com" <sourceware-bugzilla@sourceware.org> a écrit:

[...]

> --- Comment #1 from Frank Ch. Eigler <fche at redhat dot com> ---
> Created attachment 15479
>   --> https://sourceware.org/bugzilla/attachment.cgi?id=15479&action=edit
> proposed patch

Thanks for the patch!

I have massaged it to add some more logging and such.

I came up with the potential commit below that is currently running
through the build bots.  OK to apply if it passes tests?

From 3810dbe6efd487c9712c87371265e41eacc6106e Mon Sep 17 00:00:00 2001
From: "Frank Ch. Eigler" <fche@redhat.com>
Date: Fri, 26 Apr 2024 10:30:42 +0200
Subject: [PATCH] configure,abidb: Make the libarchive python module optional for abidb

Some distributions don't have the libarchive python module packaged.

This patch makes that module optional and enables abidb to function
without it, as it's only necessary for the support of the '--archive'
command line option of the abidb program.

	* configure.ac: Detect the libarchive python module as an optional
	dependency for the abidb program.  So far, only the git python
	module remains a hard dependency.
	* tools/abidb (main): Do not support the '--archive' command line
	option if the libarchive python module is not present.

Signed-off-by: Dodji Seketeli <dodji@redhat.com>
---
 configure.ac | 15 +++++++++++++--
 tools/abidb  | 14 ++++++++++----
 2 files changed, 23 insertions(+), 6 deletions(-)
  

Dodji Seketeli <dodji@seketeli.org> a écrit:

[...]

> I came up with the potential commit below that is currently running
> through the build bots.  OK to apply if it passes tests?

Actually, the previous patch failed some tests on some platform where
the libarchive python module is missing.  This is because some code in
the main() function was still referencing the archive even though it is
not supported.  Adjusted thus.

Now, the updated patch below actually passes the test on all the tested
platforms.  OK to apply to the master branch?

From f41d17301f75fab61366bd7b568418d2a9c30567 Mon Sep 17 00:00:00 2001
From: "Frank Ch. Eigler" <fche@redhat.com>
Date: Fri, 26 Apr 2024 10:30:42 +0200
Subject: [PATCH] configure,abidb: Make the libarchive python module optional for abidb

Some distributions don't have the libarchive python module packaged.

This patch makes that module optional and enables abidb to function
without it, as it's only necessary for the support of the '--archive'
command line option of the abidb program.

	* configure.ac: Detect the libarchive python module as an optional
	dependency for the abidb program.  So far, only the git python
	module remains a hard dependency.
	* tools/abidb (main): Do not support the '--archive' command line
	option if the libarchive python module is not present.

Signed-off-by: Dodji Seketeli <dodji@redhat.com>
---
 configure.ac | 15 +++++++++++++--
 tools/abidb  | 31 +++++++++++++++++++------------
 2 files changed, 32 insertions(+), 14 deletions(-)

diff --git a/configure.ac b/configure.ac
index a86f81d0..c6e1ad64 100644
--- a/configure.ac
+++ b/configure.ac
@@ -824,16 +824,27 @@ fi
 
 dnl abidb checks
 if test x$PYTHON3_INTERPRETER != xno -a x$ENABLE_ABIDB != xno; then
-  AX_CHECK_PYTHON_MODULES([git libarchive],
+  AX_CHECK_PYTHON_MODULES([git],
 			  [$PYTHON],
 			  [FOUND_ALL_PYTHON_MODULES=yes],
 			  [FOUND_ALL_PYTHON_MODULES=no])
 
   if test x$FOUND_ALL_PYTHON_MODULES = xno; then
     AC_MSG_NOTICE([missing python modules: $MISSING_PYTHON_MODULES])
-    AC_MSG_NOTICE([disabling abidb as a result])
+    AC_MSG_WARN([disabling abidb as a result])
+    ENABLE_ABIDB=no
   else
     ENABLE_ABIDB=yes
+    dnl search for optional modules, just for autoconf reporting purposes
+    AX_CHECK_PYTHON_MODULES([libarchive],
+			    [$PYTHON],
+			    [FOUND_ALL_OPTIONAL_PYTHON_MODULES=yes],
+			    [FOUND_ALL_OPTIONAL_PYTHON_MODULES=no])
+      if test x$FOUND_ALL_OPTIONAL_PYTHON_MODULES = xno; then
+        AC_MSG_WARN([missing optional python modules for abidb: $MISSING_PYTHON_MODULES])
+    else
+	AC_MSG_NOTICE([found all optional python modules for abidb])
+    fi
   fi
 else
     AC_MSG_NOTICE([disabling abidb])
diff --git a/tools/abidb b/tools/abidb
index a3f6850f..af1750d9 100755
--- a/tools/abidb
+++ b/tools/abidb
@@ -23,7 +23,11 @@ import re
 import ast
 import os
 import glob
-import libarchive
+try:
+    import libarchive
+    enable_libarchive=True
+except:
+    enable_libarchive=False
 
 
 # globals
@@ -156,9 +160,10 @@ def main() -> list[str]:
     parser.add_argument('--timeout',type=int,help='limit abidw/abicompat runtime (seconds)',default=0)
     parser.add_argument('--submit',nargs='*',type=str,default=[],
                         help='submit abidw of given binaries to abidb')
-    parser.add_argument('--archive','-Z',metavar='EXT=CMD',
-                        type=str,help='submit binaries from archives with given extension & decoder',
-                        default=[],action='append') # like debuginfod(8)
+    if enable_libarchive:
+        parser.add_argument('--archive','-Z',metavar='EXT=CMD',
+                            type=str,help='submit binaries from archives with given extension & decoder',
+                            default=[],action='append') # like debuginfod(8)
     parser.add_argument('--sysroot',type=str,help='remove given sysroot prefix from submitted file names',default=None)
     parser.add_argument('--filter',type=str,help='submit only binaries matching given wildcard',default=r'/lib.*\.so') # sub-version suffixes will be flattened into SONAME
     # --sysroot=PATH subtract this from SUBMIT paths
@@ -208,14 +213,15 @@ def main() -> list[str]:
         logging.info(f'checked out distrobranch {args.distrobranch} files {numfiles}')
 
         ra = {}
-        for entry in args.archive: # parse / accumulate -Z EXT=CMD bits
-            extcmd = entry.split('=')
-            ext = extcmd[0]
-            if len(extcmd) == 1:
-                cmd = "cat" # default: pass through to libarchive
-            else:
-                cmd = "=".join(extcmd[1:]) # the rest of the command, filling other ='s back in
-            ra["."+ext] = cmd
+        if enable_libarchive:
+            for entry in args.archive: # parse / accumulate -Z EXT=CMD bits
+                extcmd = entry.split('=')
+                ext = extcmd[0]
+                if len(extcmd) == 1:
+                    cmd = "cat" # default: pass through to libarchive
+                else:
+                    cmd = "=".join(extcmd[1:]) # the rest of the command, filling other ='s back in
+                    ra["."+ext] = cmd
 
 
         def submit_file_generator(args):
@@ -230,6 +236,7 @@ def main() -> list[str]:
                         ln = submit
                     yield (None, ln, pn) # must be a plain file # XXX or ldconfig-created symlink, ugh
                 else: # an archive!
+                    assert enable_libarchive
                     cmd = ra[ext]
                     if (cmd == "cat"): # short-circuit this
                         with libarchive.file_reader(submit) as archive:
  

Hey Frank,

Dodji Seketeli <dodji@seketeli.org> a écrit:

> Now, the updated patch below actually passes the test on all the tested
> platforms.  OK to apply to the master branch?

So, after your ACK on IRC, I went ahead and applied the patch.

> From f41d17301f75fab61366bd7b568418d2a9c30567 Mon Sep 17 00:00:00 2001
> From: "Frank Ch. Eigler" <fche@redhat.com>
> Date: Fri, 26 Apr 2024 10:30:42 +0200
> Subject: [PATCH] configure,abidb: Make the libarchive python module optional for abidb
>
> Some distributions don't have the libarchive python module packaged.
>
> This patch makes that module optional and enables abidb to function
> without it, as it's only necessary for the support of the '--archive'
> command line option of the abidb program.
>
> 	* configure.ac: Detect the libarchive python module as an optional
> 	dependency for the abidb program.  So far, only the git python
> 	module remains a hard dependency.
> 	* tools/abidb (main): Do not support the '--archive' command line
> 	option if the libarchive python module is not present.
>
> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
> ---
>  configure.ac | 15 +++++++++++++--
>  tools/abidb  | 31 +++++++++++++++++++------------
>  2 files changed, 32 insertions(+), 14 deletions(-)

[...]

Thanks!

Cheers,