[v4] Make GDB compile with Python 3 on MinGW
Commit Message
PyFile_FromString and PyFile_AsFile have been removed in Python 3.
There is no obvious replacement that works here, and we can't just
pass our FILE* to a DLL in Windows because it may use a different
C runtime.
So we just call a Python function which reads and executes file
contents. Care must be taken to execute it in the context of
__main__.
Tested by inverting the ifdef and running the testsuite on Debian
Linux (even without the patch, I failed at running the testsuite
on Windows). I did test with both Python 2 and 3.
gdb/ChangeLog:
2019-08-13 Christian Biesinger <cbiesinger@google.com>
* python/lib/gdb/__init__.py: Add an execute_file function.
* python/python.c (python_run_simple_file): Call gdb.execute_file
on Windows.
---
gdb/python/lib/gdb/__init__.py | 26 ++++++++++++++++++++++++++
gdb/python/python.c | 23 ++++++++++++++---------
2 files changed, 40 insertions(+), 9 deletions(-)
Comments
Ping
On Thu, Aug 15, 2019 at 1:49 PM Christian Biesinger
<cbiesinger@google.com> wrote:
>
> PyFile_FromString and PyFile_AsFile have been removed in Python 3.
> There is no obvious replacement that works here, and we can't just
> pass our FILE* to a DLL in Windows because it may use a different
> C runtime.
>
> So we just call a Python function which reads and executes file
> contents. Care must be taken to execute it in the context of
> __main__.
>
> Tested by inverting the ifdef and running the testsuite on Debian
> Linux (even without the patch, I failed at running the testsuite
> on Windows). I did test with both Python 2 and 3.
>
> gdb/ChangeLog:
>
> 2019-08-13 Christian Biesinger <cbiesinger@google.com>
>
> * python/lib/gdb/__init__.py: Add an execute_file function.
> * python/python.c (python_run_simple_file): Call gdb.execute_file
> on Windows.
> ---
> gdb/python/lib/gdb/__init__.py | 26 ++++++++++++++++++++++++++
> gdb/python/python.c | 23 ++++++++++++++---------
> 2 files changed, 40 insertions(+), 9 deletions(-)
>
> diff --git a/gdb/python/lib/gdb/__init__.py b/gdb/python/lib/gdb/__init__.py
> index af74df80c8..afe5b08f3a 100644
> --- a/gdb/python/lib/gdb/__init__.py
> +++ b/gdb/python/lib/gdb/__init__.py
> @@ -106,6 +106,32 @@ def execute_unwinders(pending_frame):
>
> return None
>
> +def _execute_file(filepath):
> + """This function is used to replace Python 2's PyRun_SimpleFile.
> +
> + Loads and executes the given file.
> +
> + We could use the runpy module, but its documentation says:
> + "Furthermore, any functions and classes defined by the executed code are
> + not guaranteed to work correctly after a runpy function has returned."
> + """
> + globals = sys.modules['__main__'].__dict__
> + set_file = False
> + # Set file (if not set) so that the imported file can use it (e.g. to
> + # access file-relative paths). This matches what PyRun_SimpleFile does.
> + if not hasattr(globals, '__file__'):
> + globals['__file__'] = filepath
> + set_file = True
> + try:
> + with open(filepath, 'rb') as file:
> + # We pass globals also as locals to match what Python does
> + # in PyRun_SimpleFile.
> + compiled = compile(file.read(), filepath, 'exec')
> + exec(compiled, globals, globals)
> + finally:
> + if set_file:
> + del globals['__file__']
> +
>
> # Convenience variable to GDB's python directory
> PYTHONDIR = os.path.dirname(os.path.dirname(__file__))
> diff --git a/gdb/python/python.c b/gdb/python/python.c
> index 162470dcc0..cca7c4cd6b 100644
> --- a/gdb/python/python.c
> +++ b/gdb/python/python.c
> @@ -323,9 +323,8 @@ python_interactive_command (const char *arg, int from_tty)
> A FILE * from one runtime does not necessarily operate correctly in
> the other runtime.
>
> - To work around this potential issue, we create on Windows hosts the
> - FILE object using Python routines, thus making sure that it is
> - compatible with the Python library. */
> + To work around this potential issue, we run code in Python to load
> + the script. */
>
> static void
> python_run_simple_file (FILE *file, const char *filename)
> @@ -339,15 +338,21 @@ python_run_simple_file (FILE *file, const char *filename)
> /* Because we have a string for a filename, and are using Python to
> open the file, we need to expand any tilde in the path first. */
> gdb::unique_xmalloc_ptr<char> full_path (tilde_expand (filename));
> - gdbpy_ref<> python_file (PyFile_FromString (full_path.get (), (char *) "r"));
> - if (python_file == NULL)
> +
> + if (gdb_python_module == nullptr
> + || ! PyObject_HasAttrString (gdb_python_module, "_execute_file"))
> + error (_("Installation error: gdb._execute_file function is missing"));
> +
> + gdbpy_ref<> return_value
> + (PyObject_CallMethod (gdb_python_module, "_execute_file", "s",
> + full_path.get ()));
> + if (return_value == nullptr)
> {
> - gdbpy_print_stack ();
> - error (_("Error while opening file: %s"), full_path.get ());
> + /* Use PyErr_PrintEx instead of gdbpy_print_stack to better match the
> + behavior of the non-Windows codepath. */
> + PyErr_PrintEx(0);
> }
>
> - PyRun_SimpleFile (PyFile_AsFile (python_file.get ()), filename);
> -
> #endif /* _WIN32 */
> }
>
> --
> 2.23.0.rc1.153.gdeed80330f-goog
>
On 2019-08-15 2:49 p.m., Christian Biesinger via gdb-patches wrote:
> PyFile_FromString and PyFile_AsFile have been removed in Python 3.
> There is no obvious replacement that works here, and we can't just
> pass our FILE* to a DLL in Windows because it may use a different
> C runtime.
>
> So we just call a Python function which reads and executes file
> contents. Care must be taken to execute it in the context of
> __main__.
>
> Tested by inverting the ifdef and running the testsuite on Debian
> Linux (even without the patch, I failed at running the testsuite
> on Windows). I did test with both Python 2 and 3.
>
> gdb/ChangeLog:
>
> 2019-08-13 Christian Biesinger <cbiesinger@google.com>
>
> * python/lib/gdb/__init__.py: Add an execute_file function.
This can be written as:
* python/lib/gdb/__init__.py (_execute_file): New function.
> * python/python.c (python_run_simple_file): Call gdb.execute_file
> on Windows.
Please update "execute_file" to "_execute_file" in both entries.
The patch LGTM with this fixed.
Thanks,
Simon
On Thu, Aug 22, 2019 at 5:24 PM Simon Marchi <simark@simark.ca> wrote:
>
> On 2019-08-15 2:49 p.m., Christian Biesinger via gdb-patches wrote:
> > PyFile_FromString and PyFile_AsFile have been removed in Python 3.
> > There is no obvious replacement that works here, and we can't just
> > pass our FILE* to a DLL in Windows because it may use a different
> > C runtime.
> >
> > So we just call a Python function which reads and executes file
> > contents. Care must be taken to execute it in the context of
> > __main__.
> >
> > Tested by inverting the ifdef and running the testsuite on Debian
> > Linux (even without the patch, I failed at running the testsuite
> > on Windows). I did test with both Python 2 and 3.
> >
> > gdb/ChangeLog:
> >
> > 2019-08-13 Christian Biesinger <cbiesinger@google.com>
> >
> > * python/lib/gdb/__init__.py: Add an execute_file function.
>
> This can be written as:
>
> * python/lib/gdb/__init__.py (_execute_file): New function.
>
> > * python/python.c (python_run_simple_file): Call gdb.execute_file
> > on Windows.
>
> Please update "execute_file" to "_execute_file" in both entries.
>
> The patch LGTM with this fixed.
Thanks, pushing now with that fixed.
Christian
On Thu, Aug 22, 2019 at 6:47 PM Christian Biesinger
<cbiesinger@google.com> wrote:
>
> On Thu, Aug 22, 2019 at 5:24 PM Simon Marchi <simark@simark.ca> wrote:
> >
> > On 2019-08-15 2:49 p.m., Christian Biesinger via gdb-patches wrote:
> > > PyFile_FromString and PyFile_AsFile have been removed in Python 3.
> > > There is no obvious replacement that works here, and we can't just
> > > pass our FILE* to a DLL in Windows because it may use a different
> > > C runtime.
> > >
> > > So we just call a Python function which reads and executes file
> > > contents. Care must be taken to execute it in the context of
> > > __main__.
> > >
> > > Tested by inverting the ifdef and running the testsuite on Debian
> > > Linux (even without the patch, I failed at running the testsuite
> > > on Windows). I did test with both Python 2 and 3.
[...]
> > The patch LGTM with this fixed.
>
> Thanks, pushing now with that fixed.
Thinking about this patch -- is it worth a NEWS entry to mention that
gdb can now be compiled with Python 3 on Windows?
Christian
> From: "Christian Biesinger via gdb-patches" <gdb-patches@sourceware.org>
> Date: Sat, 14 Sep 2019 09:08:51 -0400
> Cc: gdb-patches <gdb-patches@sourceware.org>
>
> Thinking about this patch -- is it worth a NEWS entry to mention that
> gdb can now be compiled with Python 3 on Windows?
You are right, it probably is.
@@ -106,6 +106,32 @@ def execute_unwinders(pending_frame):
return None
+def _execute_file(filepath):
+ """This function is used to replace Python 2's PyRun_SimpleFile.
+
+ Loads and executes the given file.
+
+ We could use the runpy module, but its documentation says:
+ "Furthermore, any functions and classes defined by the executed code are
+ not guaranteed to work correctly after a runpy function has returned."
+ """
+ globals = sys.modules['__main__'].__dict__
+ set_file = False
+ # Set file (if not set) so that the imported file can use it (e.g. to
+ # access file-relative paths). This matches what PyRun_SimpleFile does.
+ if not hasattr(globals, '__file__'):
+ globals['__file__'] = filepath
+ set_file = True
+ try:
+ with open(filepath, 'rb') as file:
+ # We pass globals also as locals to match what Python does
+ # in PyRun_SimpleFile.
+ compiled = compile(file.read(), filepath, 'exec')
+ exec(compiled, globals, globals)
+ finally:
+ if set_file:
+ del globals['__file__']
+
# Convenience variable to GDB's python directory
PYTHONDIR = os.path.dirname(os.path.dirname(__file__))
@@ -323,9 +323,8 @@ python_interactive_command (const char *arg, int from_tty)
A FILE * from one runtime does not necessarily operate correctly in
the other runtime.
- To work around this potential issue, we create on Windows hosts the
- FILE object using Python routines, thus making sure that it is
- compatible with the Python library. */
+ To work around this potential issue, we run code in Python to load
+ the script. */
static void
python_run_simple_file (FILE *file, const char *filename)
@@ -339,15 +338,21 @@ python_run_simple_file (FILE *file, const char *filename)
/* Because we have a string for a filename, and are using Python to
open the file, we need to expand any tilde in the path first. */
gdb::unique_xmalloc_ptr<char> full_path (tilde_expand (filename));
- gdbpy_ref<> python_file (PyFile_FromString (full_path.get (), (char *) "r"));
- if (python_file == NULL)
+
+ if (gdb_python_module == nullptr
+ || ! PyObject_HasAttrString (gdb_python_module, "_execute_file"))
+ error (_("Installation error: gdb._execute_file function is missing"));
+
+ gdbpy_ref<> return_value
+ (PyObject_CallMethod (gdb_python_module, "_execute_file", "s",
+ full_path.get ()));
+ if (return_value == nullptr)
{
- gdbpy_print_stack ();
- error (_("Error while opening file: %s"), full_path.get ());
+ /* Use PyErr_PrintEx instead of gdbpy_print_stack to better match the
+ behavior of the non-Windows codepath. */
+ PyErr_PrintEx(0);
}
- PyRun_SimpleFile (PyFile_AsFile (python_file.get ()), filename);
-
#endif /* _WIN32 */
}