gdb: Allow more control over where to find python libraries
Commit Message
The motivation behind this commit is to make it easier to bundle the
Python libraries with GDB when linking GDB against a static
libpython, the Python libraries will be manually added into the GDB
installation tree, and GDB should be able to find them at run-time.
The installation tree will look like this:
.
|-- bin/
|-- include/
|-- lib/
| `-- python3.8/
`-- share/
The benefit here is that the entire installation tree can be bundled
into a single archive and copied to another machine with a different
version of Python installed, and GDB will still work, including its
Python support.
In use the new configure options would be used something like this,
first build and install a static Python library:
mkdir python
cd python
# Clone or download Python into a src/ directory.
mkdir build
export PYTHON_INSTALL_PATH=$PWD/install
cd build
../src/configure --disable-shared --prefix=$PYTHON_INSTALL_PATH
make
make install
Now build and install GDB:
mkdir binutils-gdb
cd binutils-gdb
# Clone or download GDB into a src/ directory.
mkdir build
export GDB_INSTALL_DIR=$PWD/install
cd build
../src/configure \
--prefix=$GDB_INSTALL_DIR \
--with-python=$PYTHON_INSTALL_PATH/bin/python3 \
--with-python-libdir=$GDB_INSTALL_DIR/lib
make all-gdb
make install-gdb
Finally, copy the Python libraries into the GDB install:
cp -r $PYTHON_INSTALL_DIR/lib/python3.8/ $GDB_INSTALL_DIR/lib
After this the Python src, build, and install directories are no
longer needed and can be deleted.
If the new --with-python-libdir option is not used then the existing
behaviour is left unchanged, GDB will look for the Python libraries in
the lib/ directory within the python path. The concatenation of the
python prefix and the string 'lib/' is now done at configure time,
rather than at run time in GDB as it was previous, however, this was
never something that the user had dynamic control over, so there's no
loss of functionality.
gdb/ChangeLog:
* config.in: Regenerate.
* configure: Regenerate.
* configure.ac: Add --with-python-libdir option.
* main.c: Use WITH_PYTHON_LIBDIR.
Change-Id: Ic1d4d548ae3c28788f1bd7f72dfb9926b8f33de8
---
gdb/ChangeLog | 7 +++++++
gdb/config.in | 8 ++++++++
gdb/configure | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
gdb/configure.ac | 24 ++++++++++++++++++++++++
gdb/main.c | 11 +++--------
5 files changed, 91 insertions(+), 8 deletions(-)
Comments
> From: Andrew Burgess <andrew.burgess@embecosm.com>
> Cc: Richard Bunt <richard.bunt@arm.com>, Andrew Burgess <andrew.burgess@embecosm.com>
> Date: Thu, 6 Feb 2020 16:46:17 +0000
>
> The motivation behind this commit is to make it easier to bundle the
> Python libraries with GDB when linking GDB against a static
> libpython, the Python libraries will be manually added into the GDB
> installation tree, and GDB should be able to find them at run-time.
> The installation tree will look like this:
>
> .
> |-- bin/
> |-- include/
> |-- lib/
> | `-- python3.8/
> `-- share/
>
> The benefit here is that the entire installation tree can be bundled
> into a single archive and copied to another machine with a different
> version of Python installed, and GDB will still work, including its
> Python support.
This assumes that the Python libraries and support files are part of
the GDB distribution, right? But if those are distributed with GDB,
so should be their sources, to adhere to the license, no?
On 2020-02-06 11:46 a.m., Andrew Burgess wrote:
> The motivation behind this commit is to make it easier to bundle the
> Python libraries with GDB when linking GDB against a static
> libpython, the Python libraries will be manually added into the GDB
> installation tree, and GDB should be able to find them at run-time.
> The installation tree will look like this:
>
> .
> |-- bin/
> |-- include/
> |-- lib/
> | `-- python3.8/
> `-- share/
>
> The benefit here is that the entire installation tree can be bundled
> into a single archive and copied to another machine with a different
> version of Python installed, and GDB will still work, including its
> Python support.
For those who might be wondering, like me: isn't the goal of linking
statically with the lib to avoid having an external library? This
patch actually deals with finding the .py files provided by the Python
standard library, not the native code. The native code is indeed
linked statically inside the gdb executable.
I won't pretend to understand in details what's happening with the Python
detection in configure (I find it's quite messy), but I didn't spot anything
wrong with your patch, and it seems to address a valid use case, so I'm not
against it.
Simon
* Simon Marchi <simark@simark.ca> [2020-02-07 12:58:46 -0500]:
> On 2020-02-06 11:46 a.m., Andrew Burgess wrote:
> > The motivation behind this commit is to make it easier to bundle the
> > Python libraries with GDB when linking GDB against a static
> > libpython, the Python libraries will be manually added into the GDB
> > installation tree, and GDB should be able to find them at run-time.
> > The installation tree will look like this:
> >
> > .
> > |-- bin/
> > |-- include/
> > |-- lib/
> > | `-- python3.8/
> > `-- share/
> >
> > The benefit here is that the entire installation tree can be bundled
> > into a single archive and copied to another machine with a different
> > version of Python installed, and GDB will still work, including its
> > Python support.
>
> For those who might be wondering, like me: isn't the goal of linking
> statically with the lib to avoid having an external library? This
> patch actually deals with finding the .py files provided by the Python
> standard library, not the native code. The native code is indeed
> linked statically inside the gdb executable.
Yes, this is it exactly. I will rewrite the commit message to say '.py
files' instead as that's much clearer.
Thanks,
Andrew
>
> I won't pretend to understand in details what's happening with the Python
> detection in configure (I find it's quite messy), but I didn't spot anything
> wrong with your patch, and it seems to address a valid use case, so I'm not
> against it.
>
> Simon
* Eli Zaretskii <eliz@gnu.org> [2020-02-06 20:31:23 +0200]:
> > From: Andrew Burgess <andrew.burgess@embecosm.com>
> > Cc: Richard Bunt <richard.bunt@arm.com>, Andrew Burgess <andrew.burgess@embecosm.com>
> > Date: Thu, 6 Feb 2020 16:46:17 +0000
> >
> > The motivation behind this commit is to make it easier to bundle the
> > Python libraries with GDB when linking GDB against a static
> > libpython, the Python libraries will be manually added into the GDB
> > installation tree, and GDB should be able to find them at run-time.
> > The installation tree will look like this:
> >
> > .
> > |-- bin/
> > |-- include/
> > |-- lib/
> > | `-- python3.8/
> > `-- share/
> >
> > The benefit here is that the entire installation tree can be bundled
> > into a single archive and copied to another machine with a different
> > version of Python installed, and GDB will still work, including its
> > Python support.
>
> This assumes that the Python libraries and support files are part of
> the GDB distribution, right? But if those are distributed with GDB,
> so should be their sources, to adhere to the license, no?
Hi Eli,
As Simon mentioned I don't think that I described my intentions very
well. The intended use case for this situation is building the Python
interpreter statically into GDB, and then placing Python's *.py files
into a directory relative to the built GDB executable, and have GDB
manage to find them.
The use for this would be that at XXX organisation I can build a
version of GDB, package it up into a tar-file and copy this onto
several different machines, which might be running different OS
versions.
In this situation I don't think there's any licensing issue as the
builds of GDB are not going outside the XXX organisation.
If I did decide to distribute the pre-built GDB tar-files outside of
XXX, then the source for GDB, and the source for Python would be made
available also from XXX, but I didn't believe simply distributing two
pre-built things in one package means I have to upstream merge the two
projects - have I miss-understood?
Thanks,
Andrew
> Date: Wed, 19 Feb 2020 15:53:50 +0000
> From: Andrew Burgess <andrew.burgess@embecosm.com>
> Cc: gdb-patches@sourceware.org, richard.bunt@arm.com
>
> The use for this would be that at XXX organisation I can build a
> version of GDB, package it up into a tar-file and copy this onto
> several different machines, which might be running different OS
> versions.
>
> In this situation I don't think there's any licensing issue as the
> builds of GDB are not going outside the XXX organisation.
>
> If I did decide to distribute the pre-built GDB tar-files outside of
> XXX, then the source for GDB, and the source for Python would be made
> available also from XXX, but I didn't believe simply distributing two
> pre-built things in one package means I have to upstream merge the two
> projects - have I miss-understood?
If this is for private use between you and yourself, more or less,
then there's no problem, indeed.
@@ -649,6 +649,10 @@
/* Define as the return type of ptrace. */
#undef PTRACE_TYPE_RET
+/* Define if the python lib directory should be relocated when GDB is moved.
+ */
+#undef PYTHON_LIBDIR_RELOCATABLE
+
/* Define if the python directory should be relocated when GDB is moved. */
#undef PYTHON_PATH_RELOCATABLE
@@ -752,6 +756,10 @@
/* Define if the PPC simulator is being linked in. */
#undef WITH_PPC_SIM
+/* Directory containing Python's standard libraries from --with-python-libdir.
+ */
+#undef WITH_PYTHON_LIBDIR
+
/* Define if --with-python provides a path, either directly or via
python-config.py --exec-prefix. */
#undef WITH_PYTHON_PATH
@@ -887,6 +887,7 @@ with_libexpat_prefix
with_mpfr
with_libmpfr_prefix
with_python
+with_python_libdir
with_guile
enable_source_highlight
with_intel_pt
@@ -1619,6 +1620,8 @@ Optional Packages:
--without-libmpfr-prefix don't search for libmpfr in includedir and libdir
--with-python[=PYTHON] include python support
(auto/yes/no/<python-program>)
+ --with-python-libdir[=DIR]
+ search for python's libraries in DIR
--with-guile[=GUILE] include guile support
(auto/yes/no/<guile-version>/<pkg-config-program>)
--with-intel-pt include Intel Processor Trace support (auto/yes/no)
@@ -10611,6 +10614,21 @@ _ACEOF
fi
fi
+
+# Check whether --with-python-libdir was given.
+if test "${with_python_libdir+set}" = set; then :
+ withval=$with_python_libdir;
+else
+
+ # If no python libdir is specified then select one based on
+ # python's prefix path.
+ if test -n "${python_prefix}"; then
+ with_python_libdir=${python_prefix}/lib
+ fi
+
+fi
+
+
if test "${have_libpython}" != no; then
$as_echo "#define HAVE_PYTHON 1" >>confdefs.h
@@ -10621,6 +10639,37 @@ $as_echo "#define HAVE_PYTHON 1" >>confdefs.h
CONFIG_INSTALL="$CONFIG_INSTALL install-python"
ENABLE_CFLAGS="$ENABLE_CFLAGS \$(SUBDIR_PYTHON_CFLAGS)"
+ if test -n "${with_python_libdir}"; then
+
+cat >>confdefs.h <<_ACEOF
+#define WITH_PYTHON_LIBDIR "${with_python_libdir}"
+_ACEOF
+
+
+ if test "x$exec_prefix" = xNONE || test "x$exec_prefix" = 'x${prefix}'; then
+ if test "x$prefix" = xNONE; then
+ test_prefix=/usr/local
+ else
+ test_prefix=$prefix
+ fi
+ else
+ test_prefix=$exec_prefix
+ fi
+ value=0
+ case ${with_python_libdir} in
+ "${test_prefix}"|"${test_prefix}/"*|\
+ '${exec_prefix}'|'${exec_prefix}/'*)
+ value=1
+ ;;
+ esac
+
+cat >>confdefs.h <<_ACEOF
+#define PYTHON_LIBDIR_RELOCATABLE $value
+_ACEOF
+
+
+ fi
+
# Flags needed to compile Python code (taken from python-config --cflags).
# We cannot call python-config directly because it will output whatever was
# used when compiling the Python interpreter itself, including flags which
@@ -891,6 +891,24 @@ else
fi
fi
+dnl Use --with-python-libdir to control where GDB looks for the Python
+dnl libraries.
+dnl
+dnl If this is not given then the default will be based on the value
+dnl passed to --with-python, which is in the python_prefix variable.
+dnl If the --with-python option wasn't given then the default value in
+dnl python_prefix is based on running the 'gdb/python/python-config
+dnl --exec-prefix' script.
+AC_ARG_WITH(python-libdir,
+ AS_HELP_STRING([--with-python-libdir@<:@=DIR@:>@], [search for python's libraries in DIR]),
+ [],[
+ # If no python libdir is specified then select one based on
+ # python's prefix path.
+ if test -n "${python_prefix}"; then
+ with_python_libdir=${python_prefix}/lib
+ fi
+ ])
+
if test "${have_libpython}" != no; then
AC_DEFINE(HAVE_PYTHON, 1, [Define if Python interpreter is being linked in.])
CONFIG_OBS="$CONFIG_OBS \$(SUBDIR_PYTHON_OBS)"
@@ -899,6 +917,12 @@ if test "${have_libpython}" != no; then
CONFIG_INSTALL="$CONFIG_INSTALL install-python"
ENABLE_CFLAGS="$ENABLE_CFLAGS \$(SUBDIR_PYTHON_CFLAGS)"
+ if test -n "${with_python_libdir}"; then
+ AC_DEFINE_UNQUOTED(WITH_PYTHON_LIBDIR, "${with_python_libdir}",
+ [Directory containing Python's standard libraries from --with-python-libdir.])
+ GDB_AC_DEFINE_RELOCATABLE(PYTHON_LIBDIR, [python lib], ${with_python_libdir})
+ fi
+
# Flags needed to compile Python code (taken from python-config --cflags).
# We cannot call python-config directly because it will output whatever was
# used when compiling the Python interpreter itself, including flags which
@@ -575,14 +575,9 @@ captured_main_1 (struct captured_main_args *context)
gdb_datadir = relocate_gdb_directory (GDB_DATADIR,
GDB_DATADIR_RELOCATABLE);
-#ifdef WITH_PYTHON_PATH
- {
- /* For later use in helping Python find itself. */
- char *tmp = concat (WITH_PYTHON_PATH, SLASH_STRING, "lib", (char *) NULL);
-
- python_libdir = relocate_gdb_directory (tmp, PYTHON_PATH_RELOCATABLE);
- xfree (tmp);
- }
+#ifdef WITH_PYTHON_LIBDIR
+ python_libdir = relocate_gdb_directory (WITH_PYTHON_LIBDIR,
+ PYTHON_LIBDIR_RELOCATABLE);
#endif
#ifdef RELOC_SRCDIR