| Message ID | 20200206164617.7461-1-andrew.burgess@embecosm.com |
|---|---|
| State | New |
| Headers | show |
> From: Andrew Burgess <andrew.burgess@embecosm.com> > Cc: Richard Bunt <richard.bunt@arm.com>, Andrew Burgess <andrew.burgess@embecosm.com> > Date: Thu, 6 Feb 2020 16:46:17 +0000 > > The motivation behind this commit is to make it easier to bundle the > Python libraries with GDB when linking GDB against a static > libpython, the Python libraries will be manually added into the GDB > installation tree, and GDB should be able to find them at run-time. > The installation tree will look like this: > > . > |-- bin/ > |-- include/ > |-- lib/ > | `-- python3.8/ > `-- share/ > > The benefit here is that the entire installation tree can be bundled > into a single archive and copied to another machine with a different > version of Python installed, and GDB will still work, including its > Python support. This assumes that the Python libraries and support files are part of the GDB distribution, right? But if those are distributed with GDB, so should be their sources, to adhere to the license, no?
On 2020-02-06 11:46 a.m., Andrew Burgess wrote: > The motivation behind this commit is to make it easier to bundle the > Python libraries with GDB when linking GDB against a static > libpython, the Python libraries will be manually added into the GDB > installation tree, and GDB should be able to find them at run-time. > The installation tree will look like this: > > . > |-- bin/ > |-- include/ > |-- lib/ > | `-- python3.8/ > `-- share/ > > The benefit here is that the entire installation tree can be bundled > into a single archive and copied to another machine with a different > version of Python installed, and GDB will still work, including its > Python support. For those who might be wondering, like me: isn't the goal of linking statically with the lib to avoid having an external library? This patch actually deals with finding the .py files provided by the Python standard library, not the native code. The native code is indeed linked statically inside the gdb executable. I won't pretend to understand in details what's happening with the Python detection in configure (I find it's quite messy), but I didn't spot anything wrong with your patch, and it seems to address a valid use case, so I'm not against it. Simon
* Simon Marchi <simark@simark.ca> [2020-02-07 12:58:46 -0500]: > On 2020-02-06 11:46 a.m., Andrew Burgess wrote: > > The motivation behind this commit is to make it easier to bundle the > > Python libraries with GDB when linking GDB against a static > > libpython, the Python libraries will be manually added into the GDB > > installation tree, and GDB should be able to find them at run-time. > > The installation tree will look like this: > > > > . > > |-- bin/ > > |-- include/ > > |-- lib/ > > | `-- python3.8/ > > `-- share/ > > > > The benefit here is that the entire installation tree can be bundled > > into a single archive and copied to another machine with a different > > version of Python installed, and GDB will still work, including its > > Python support. > > For those who might be wondering, like me: isn't the goal of linking > statically with the lib to avoid having an external library? This > patch actually deals with finding the .py files provided by the Python > standard library, not the native code. The native code is indeed > linked statically inside the gdb executable. Yes, this is it exactly. I will rewrite the commit message to say '.py files' instead as that's much clearer. Thanks, Andrew > > I won't pretend to understand in details what's happening with the Python > detection in configure (I find it's quite messy), but I didn't spot anything > wrong with your patch, and it seems to address a valid use case, so I'm not > against it. > > Simon
* Eli Zaretskii <eliz@gnu.org> [2020-02-06 20:31:23 +0200]: > > From: Andrew Burgess <andrew.burgess@embecosm.com> > > Cc: Richard Bunt <richard.bunt@arm.com>, Andrew Burgess <andrew.burgess@embecosm.com> > > Date: Thu, 6 Feb 2020 16:46:17 +0000 > > > > The motivation behind this commit is to make it easier to bundle the > > Python libraries with GDB when linking GDB against a static > > libpython, the Python libraries will be manually added into the GDB > > installation tree, and GDB should be able to find them at run-time. > > The installation tree will look like this: > > > > . > > |-- bin/ > > |-- include/ > > |-- lib/ > > | `-- python3.8/ > > `-- share/ > > > > The benefit here is that the entire installation tree can be bundled > > into a single archive and copied to another machine with a different > > version of Python installed, and GDB will still work, including its > > Python support. > > This assumes that the Python libraries and support files are part of > the GDB distribution, right? But if those are distributed with GDB, > so should be their sources, to adhere to the license, no? Hi Eli, As Simon mentioned I don't think that I described my intentions very well. The intended use case for this situation is building the Python interpreter statically into GDB, and then placing Python's *.py files into a directory relative to the built GDB executable, and have GDB manage to find them. The use for this would be that at XXX organisation I can build a version of GDB, package it up into a tar-file and copy this onto several different machines, which might be running different OS versions. In this situation I don't think there's any licensing issue as the builds of GDB are not going outside the XXX organisation. If I did decide to distribute the pre-built GDB tar-files outside of XXX, then the source for GDB, and the source for Python would be made available also from XXX, but I didn't believe simply distributing two pre-built things in one package means I have to upstream merge the two projects - have I miss-understood? Thanks, Andrew
> Date: Wed, 19 Feb 2020 15:53:50 +0000 > From: Andrew Burgess <andrew.burgess@embecosm.com> > Cc: gdb-patches@sourceware.org, richard.bunt@arm.com > > The use for this would be that at XXX organisation I can build a > version of GDB, package it up into a tar-file and copy this onto > several different machines, which might be running different OS > versions. > > In this situation I don't think there's any licensing issue as the > builds of GDB are not going outside the XXX organisation. > > If I did decide to distribute the pre-built GDB tar-files outside of > XXX, then the source for GDB, and the source for Python would be made > available also from XXX, but I didn't believe simply distributing two > pre-built things in one package means I have to upstream merge the two > projects - have I miss-understood? If this is for private use between you and yourself, more or less, then there's no problem, indeed.
diff --git a/gdb/config.in b/gdb/config.in index cb886ba8e1a..9c5c1ce834d 100644 --- a/gdb/config.in +++ b/gdb/config.in @@ -649,6 +649,10 @@ /* Define as the return type of ptrace. */ #undef PTRACE_TYPE_RET +/* Define if the python lib directory should be relocated when GDB is moved. + */ +#undef PYTHON_LIBDIR_RELOCATABLE + /* Define if the python directory should be relocated when GDB is moved. */ #undef PYTHON_PATH_RELOCATABLE @@ -752,6 +756,10 @@ /* Define if the PPC simulator is being linked in. */ #undef WITH_PPC_SIM +/* Directory containing Python's standard libraries from --with-python-libdir. + */ +#undef WITH_PYTHON_LIBDIR + /* Define if --with-python provides a path, either directly or via python-config.py --exec-prefix. */ #undef WITH_PYTHON_PATH diff --git a/gdb/configure b/gdb/configure index 72ffad8d37b..a595ca3512b 100755 --- a/gdb/configure +++ b/gdb/configure @@ -887,6 +887,7 @@ with_libexpat_prefix with_mpfr with_libmpfr_prefix with_python +with_python_libdir with_guile enable_source_highlight with_intel_pt @@ -1619,6 +1620,8 @@ Optional Packages: --without-libmpfr-prefix don't search for libmpfr in includedir and libdir --with-python[=PYTHON] include python support (auto/yes/no/<python-program>) + --with-python-libdir[=DIR] + search for python's libraries in DIR --with-guile[=GUILE] include guile support (auto/yes/no/<guile-version>/<pkg-config-program>) --with-intel-pt include Intel Processor Trace support (auto/yes/no) @@ -10611,6 +10614,21 @@ _ACEOF fi fi + +# Check whether --with-python-libdir was given. +if test "${with_python_libdir+set}" = set; then : + withval=$with_python_libdir; +else + + # If no python libdir is specified then select one based on + # python's prefix path. + if test -n "${python_prefix}"; then + with_python_libdir=${python_prefix}/lib + fi + +fi + + if test "${have_libpython}" != no; then $as_echo "#define HAVE_PYTHON 1" >>confdefs.h @@ -10621,6 +10639,37 @@ $as_echo "#define HAVE_PYTHON 1" >>confdefs.h CONFIG_INSTALL="$CONFIG_INSTALL install-python" ENABLE_CFLAGS="$ENABLE_CFLAGS \$(SUBDIR_PYTHON_CFLAGS)" + if test -n "${with_python_libdir}"; then + +cat >>confdefs.h <<_ACEOF +#define WITH_PYTHON_LIBDIR "${with_python_libdir}" +_ACEOF + + + if test "x$exec_prefix" = xNONE || test "x$exec_prefix" = 'x${prefix}'; then + if test "x$prefix" = xNONE; then + test_prefix=/usr/local + else + test_prefix=$prefix + fi + else + test_prefix=$exec_prefix + fi + value=0 + case ${with_python_libdir} in + "${test_prefix}"|"${test_prefix}/"*|\ + '${exec_prefix}'|'${exec_prefix}/'*) + value=1 + ;; + esac + +cat >>confdefs.h <<_ACEOF +#define PYTHON_LIBDIR_RELOCATABLE $value +_ACEOF + + + fi + # Flags needed to compile Python code (taken from python-config --cflags). # We cannot call python-config directly because it will output whatever was # used when compiling the Python interpreter itself, including flags which diff --git a/gdb/configure.ac b/gdb/configure.ac index 0ca169101b3..38a4e5a04b7 100644 --- a/gdb/configure.ac +++ b/gdb/configure.ac @@ -891,6 +891,24 @@ else fi fi +dnl Use --with-python-libdir to control where GDB looks for the Python +dnl libraries. +dnl +dnl If this is not given then the default will be based on the value +dnl passed to --with-python, which is in the python_prefix variable. +dnl If the --with-python option wasn't given then the default value in +dnl python_prefix is based on running the 'gdb/python/python-config +dnl --exec-prefix' script. +AC_ARG_WITH(python-libdir, + AS_HELP_STRING([--with-python-libdir@<:@=DIR@:>@], [search for python's libraries in DIR]), + [],[ + # If no python libdir is specified then select one based on + # python's prefix path. + if test -n "${python_prefix}"; then + with_python_libdir=${python_prefix}/lib + fi + ]) + if test "${have_libpython}" != no; then AC_DEFINE(HAVE_PYTHON, 1, [Define if Python interpreter is being linked in.]) CONFIG_OBS="$CONFIG_OBS \$(SUBDIR_PYTHON_OBS)" @@ -899,6 +917,12 @@ if test "${have_libpython}" != no; then CONFIG_INSTALL="$CONFIG_INSTALL install-python" ENABLE_CFLAGS="$ENABLE_CFLAGS \$(SUBDIR_PYTHON_CFLAGS)" + if test -n "${with_python_libdir}"; then + AC_DEFINE_UNQUOTED(WITH_PYTHON_LIBDIR, "${with_python_libdir}", + [Directory containing Python's standard libraries from --with-python-libdir.]) + GDB_AC_DEFINE_RELOCATABLE(PYTHON_LIBDIR, [python lib], ${with_python_libdir}) + fi + # Flags needed to compile Python code (taken from python-config --cflags). # We cannot call python-config directly because it will output whatever was # used when compiling the Python interpreter itself, including flags which diff --git a/gdb/main.c b/gdb/main.c index d5e5a678baa..a03ed8117ab 100644 --- a/gdb/main.c +++ b/gdb/main.c @@ -575,14 +575,9 @@ captured_main_1 (struct captured_main_args *context) gdb_datadir = relocate_gdb_directory (GDB_DATADIR, GDB_DATADIR_RELOCATABLE); -#ifdef WITH_PYTHON_PATH - { - /* For later use in helping Python find itself. */ - char *tmp = concat (WITH_PYTHON_PATH, SLASH_STRING, "lib", (char *) NULL); - - python_libdir = relocate_gdb_directory (tmp, PYTHON_PATH_RELOCATABLE); - xfree (tmp); - } +#ifdef WITH_PYTHON_LIBDIR + python_libdir = relocate_gdb_directory (WITH_PYTHON_LIBDIR, + PYTHON_LIBDIR_RELOCATABLE); #endif #ifdef RELOC_SRCDIR
The motivation behind this commit is to make it easier to bundle the Python libraries with GDB when linking GDB against a static libpython, the Python libraries will be manually added into the GDB installation tree, and GDB should be able to find them at run-time. The installation tree will look like this: . |-- bin/ |-- include/ |-- lib/ | `-- python3.8/ `-- share/ The benefit here is that the entire installation tree can be bundled into a single archive and copied to another machine with a different version of Python installed, and GDB will still work, including its Python support. In use the new configure options would be used something like this, first build and install a static Python library: mkdir python cd python # Clone or download Python into a src/ directory. mkdir build export PYTHON_INSTALL_PATH=$PWD/install cd build ../src/configure --disable-shared --prefix=$PYTHON_INSTALL_PATH make make install Now build and install GDB: mkdir binutils-gdb cd binutils-gdb # Clone or download GDB into a src/ directory. mkdir build export GDB_INSTALL_DIR=$PWD/install cd build ../src/configure \ --prefix=$GDB_INSTALL_DIR \ --with-python=$PYTHON_INSTALL_PATH/bin/python3 \ --with-python-libdir=$GDB_INSTALL_DIR/lib make all-gdb make install-gdb Finally, copy the Python libraries into the GDB install: cp -r $PYTHON_INSTALL_DIR/lib/python3.8/ $GDB_INSTALL_DIR/lib After this the Python src, build, and install directories are no longer needed and can be deleted. If the new --with-python-libdir option is not used then the existing behaviour is left unchanged, GDB will look for the Python libraries in the lib/ directory within the python path. The concatenation of the python prefix and the string 'lib/' is now done at configure time, rather than at run time in GDB as it was previous, however, this was never something that the user had dynamic control over, so there's no loss of functionality. gdb/ChangeLog: * config.in: Regenerate. * configure: Regenerate. * configure.ac: Add --with-python-libdir option. * main.c: Use WITH_PYTHON_LIBDIR. Change-Id: Ic1d4d548ae3c28788f1bd7f72dfb9926b8f33de8 --- gdb/ChangeLog | 7 +++++++ gdb/config.in | 8 ++++++++ gdb/configure | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ gdb/configure.ac | 24 ++++++++++++++++++++++++ gdb/main.c | 11 +++-------- 5 files changed, 91 insertions(+), 8 deletions(-)