linux: Add maintenance commands to test libthread_db
Commit Message
This commit adds two new commands which may be used to test thread
debugging libraries used by GDB:
* "maint check libthread-db" tests the thread debugging library GDB
is using for the current inferior.
* "maint set/show check-libthread-db" selects whether libthread_db
tests should be run automatically as libthread_db is auto-loaded.
The default is to not run tests automatically.
The test itself is a basic integrity check exercising all libthread_db
functions used by GDB on GNU/Linux systems. By extension this also
exercises the proc_service functions provided by GDB that libthread_db
uses.
This functionality is useful for NPTL developers and libthread_db
developers. It could also prove useful investigating bugs reported
against GDB where the thread debugging library or GDB's proc_service
layer is suspect.
Built and regtested on RHEL 7.4 x86_64.
Ok to commit?
Thanks,
Gary
--
gdb/ChangeLog:
* linux-thread-db.c (valprint.h): New include.
(struct check_thread_db_info): New structure.
(check_thread_db_on_load, tdb_testinfo): New static globals.
(check_thread_db, check_thread_db_callback): New functions.
(try_thread_db_load_1): Run integrity checks if requested.
(maintenance_check_libthread_db): New function.
(_initialize_thread_db): Register "maint check libthread-db"
and "maint set/show check-libthread-db".
gdb/doc/ChangeLog:
* gdb.texinfo (Maintenance Commands): Document "maint check
libthread-db" and "maint set/show check-libthread-db".
gdb/testsuite/ChangeLog:
* gdb.threads/check-libthread-db.exp: New file.
* gdb.threads/check-libthread-db.c: Likewise.
---
gdb/ChangeLog | 11 +
gdb/doc/ChangeLog | 5 +
gdb/doc/gdb.texinfo | 13 ++
gdb/linux-thread-db.c | 257 +++++++++++++++++++++++
gdb/testsuite/ChangeLog | 5 +
gdb/testsuite/gdb.threads/check-libthread-db.c | 66 ++++++
gdb/testsuite/gdb.threads/check-libthread-db.exp | 114 ++++++++++
7 files changed, 471 insertions(+)
create mode 100644 gdb/testsuite/gdb.threads/check-libthread-db.c
create mode 100644 gdb/testsuite/gdb.threads/check-libthread-db.exp
Comments
On 2017-11-22 09:42 AM, Gary Benson wrote:
> This commit adds two new commands which may be used to test thread
> debugging libraries used by GDB:
>
> * "maint check libthread-db" tests the thread debugging library GDB
> is using for the current inferior.
>
> * "maint set/show check-libthread-db" selects whether libthread_db
> tests should be run automatically as libthread_db is auto-loaded.
> The default is to not run tests automatically.
>
> The test itself is a basic integrity check exercising all libthread_db
> functions used by GDB on GNU/Linux systems. By extension this also
> exercises the proc_service functions provided by GDB that libthread_db
> uses.
>
> This functionality is useful for NPTL developers and libthread_db
> developers. It could also prove useful investigating bugs reported
> against GDB where the thread debugging library or GDB's proc_service
> layer is suspect.
>
> Built and regtested on RHEL 7.4 x86_64.
>
> Ok to commit?
>
> Thanks,
> Gary
Hi Gary,
You have much more experience than I do in this domain, so I'll trust
you if you say it's useful :). I noted a few comments below, nothing
major.
I would be fine with the patch being committed with those fixed, but
maybe give the others a week or so if they want to add comment (including
Eli for the doc). If you don't hear anything more after a week, please
go ahead and push.
>
> --
> gdb/ChangeLog:
>
> * linux-thread-db.c (valprint.h): New include.
> (struct check_thread_db_info): New structure.
> (check_thread_db_on_load, tdb_testinfo): New static globals.
> (check_thread_db, check_thread_db_callback): New functions.
> (try_thread_db_load_1): Run integrity checks if requested.
> (maintenance_check_libthread_db): New function.
> (_initialize_thread_db): Register "maint check libthread-db"
> and "maint set/show check-libthread-db".
>
> gdb/doc/ChangeLog:
>
> * gdb.texinfo (Maintenance Commands): Document "maint check
> libthread-db" and "maint set/show check-libthread-db".
>
> gdb/testsuite/ChangeLog:
>
> * gdb.threads/check-libthread-db.exp: New file.
> * gdb.threads/check-libthread-db.c: Likewise.
> ---
> gdb/ChangeLog | 11 +
> gdb/doc/ChangeLog | 5 +
> gdb/doc/gdb.texinfo | 13 ++
> gdb/linux-thread-db.c | 257 +++++++++++++++++++++++
> gdb/testsuite/ChangeLog | 5 +
> gdb/testsuite/gdb.threads/check-libthread-db.c | 66 ++++++
> gdb/testsuite/gdb.threads/check-libthread-db.exp | 114 ++++++++++
> 7 files changed, 471 insertions(+)
> create mode 100644 gdb/testsuite/gdb.threads/check-libthread-db.c
> create mode 100644 gdb/testsuite/gdb.threads/check-libthread-db.exp
>
> diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
> index 29d4789..4d7a7b1 100644
> --- a/gdb/doc/gdb.texinfo
> +++ b/gdb/doc/gdb.texinfo
> @@ -34981,6 +34981,11 @@ modify XML target descriptions.
> Check that the target descriptions dynamically created by @value{GDBN}
> equal the descriptions created from XML files found in @var{dir}.
>
> +@kindex maint check libthread-db
> +@item maint check libthread-db
> +Run integrity checks on the current inferior's thread debugging
> +library.
> +
> @kindex maint print dummy-frames
> @item maint print dummy-frames
> Prints the contents of @value{GDBN}'s internal dummy-frame stack.
> @@ -35288,6 +35293,14 @@ number of blocks in the blockvector
> @end enumerate
> @end table
>
> +@kindex maint set check-libthread-db
> +@kindex maint show check-libthread-db
> +@item maint set check-libthread-db [on|off]
> +@itemx maint show check-libthread-db
> +Control whether @value{GDBN} should run integrity checks on inferior
> +specific thread debugging libraries as they are loaded. The default
> +is not to perform such checks.
> +
> @kindex maint space
> @cindex memory used by commands
> @item maint space @var{value}
> diff --git a/gdb/linux-thread-db.c b/gdb/linux-thread-db.c
> index ea032fc..d1fdb33 100644
> --- a/gdb/linux-thread-db.c
> +++ b/gdb/linux-thread-db.c
> @@ -46,6 +46,7 @@
> #include <ctype.h>
> #include "nat/linux-namespaces.h"
> #include <algorithm>
> +#include "valprint.h"
>
> /* GNU/Linux libthread_db support.
>
> @@ -80,6 +81,10 @@ static char *libthread_db_search_path;
> by the "set auto-load libthread-db" command. */
> static int auto_load_thread_db = 1;
>
> +/* Set to non-zero if load-time libthread_db tests have been enabled
> + by the "maintenence set check-libthread-db" command. */
"maintenence".
> +static int check_thread_db_on_load = 0;
> +
> /* "show" command for the auto_load_thread_db configuration variable. */
>
> static void
> @@ -493,6 +498,216 @@ dladdr_to_soname (const void *addr)
> return NULL;
> }
>
> +/* State for check_thread_db_callback. */
> +
> +struct check_thread_db_info
> +{
> + /* The libthread_db under test. */
> + struct thread_db_info *info;
> +
> + /* True if progress should be logged. */
> + bool log_progress;
> +
> + /* True if the callback was called. */
> + bool threads_seen;
> +};
> +
> +static struct check_thread_db_info *tdb_testinfo;
>
> +
> +/* Callback for check_thread_db. */
> +
> +static int
> +check_thread_db_callback (const td_thrhandle_t *th, void *arg)
> +{
> + gdb_assert (tdb_testinfo != NULL);
> + tdb_testinfo->threads_seen = true;
> +
> +#define LOG(fmt, args...) \
> + do \
> + { \
> + if (tdb_testinfo->log_progress) \
> + { \
> + debug_printf (fmt, ## args); \
> + gdb_flush (gdb_stdlog); \
> + } \
> + } \
> + while (0)
> +
> +#define __CHECK(expr, args...) \
> + do \
> + { \
> + if (!(expr)) \
> + { \
> + LOG (" ... FAIL!\n"); \
> + error (args); \
> + } \
> + } \
> + while (0)
> +
> +#define CHECK(expr) \
> + __CHECK (expr, "(%s) == false", #expr)
> +
> +#define CHECK_CALL(func, args...) \
> + do \
> + { \
> + td_err_e __err = tdb_testinfo->info->func ## _p (args); \
> + \
> + __CHECK (__err == TD_OK, _("%s failed: %s"), #func, \
> + thread_db_err_str (__err)); \
> + } \
> + while (0)
> +
> + LOG (" Got thread");
> +
> + /* Check td_ta_thr_iter passed consistent arguments. */
> + CHECK (th != NULL);
> + CHECK (arg == (void *) tdb_testinfo);
> + CHECK (th->th_ta_p == tdb_testinfo->info->thread_agent);
> +
> + LOG (" %s", core_addr_to_string_nz ((CORE_ADDR) th->th_unique));
> +
> + /* Check td_thr_get_info. */
> + td_thrinfo_t ti;
> + CHECK_CALL (td_thr_get_info, th, &ti);
> +
> + LOG (" => %d", ti.ti_lid);
> +
> + CHECK (ti.ti_ta_p == th->th_ta_p);
> + CHECK (ti.ti_tid == (thread_t) th->th_unique);
> +
> + /* Check td_ta_map_lwp2thr. */
> + td_thrhandle_t th2;
> + memset (&th2, 23, sizeof (td_thrhandle_t));
> + CHECK_CALL (td_ta_map_lwp2thr, th->th_ta_p, ti.ti_lid, &th2);
> +
> + LOG (" => %s", core_addr_to_string_nz ((CORE_ADDR) th2.th_unique));
> +
> + CHECK (memcmp (th, &th2, sizeof (td_thrhandle_t)) == 0);
> +
> + /* Attempt TLS access. Assuming errno is TLS, this calls
> + thread_db_get_thread_local_address, which in turn calls
> + td_thr_tls_get_addr for live inferiors or td_thr_tlsbase
> + for core files. This test is skipped if the thread has
> + not been recorded; proceeding in that case would result
> + in the test having the side-effect of noticing threads
> + which seems wrong.
> +
> + Note that in glibc's libthread_db td_thr_tls_get_addr is
> + a thin wrapper around td_thr_tlsbase; this check always
> + hits the bulk of the code.
> +
> + Note also that we don't actually check any libthread_db
> + calls are made, we just assume they were; future changes
> + to how GDB accesses TLS could result in this passing
> + without exercising the calls it's supposed to. */
> + ptid_t ptid = ptid_build (tdb_testinfo->info->pid, ti.ti_lid, 0);
> + struct thread_info *thread_info = find_thread_ptid (ptid);
> + if (thread_info != NULL && thread_info->priv != NULL)
> + {
> + LOG ("; errno");
> +
> + scoped_restore_current_thread restore_current_thread;
> + switch_to_thread (ptid);
> +
> + expression_up expr = parse_expression ("errno");
Without debug symbols for libc, I get this
(gdb) maintenance check libthread-db
Running libthread_db integrity checks:
Got thread 0x7ffff7fc0700 => 7975 => 0x7ffff7fc0700; errnowarning: libthread_db integrity checks failed: 'errno' has unknown type; cast it to its declared type
thanks to Pedro's no-debuginfo changes :). I think it could easily be
worked around with
- expression_up expr = parse_expression ("errno");
+ expression_up expr = parse_expression ("(int) errno");
so that the test works even without libc debug info. With that change, I
get this, which looks good:
(gdb) maintenance check libthread-db
Running libthread_db integrity checks:
Got thread 0x7ffff7fc0700 => 8928 => 0x7ffff7fc0700; errno = 0 ... OK
libthread_db integrity checks passed.
> + struct value *val = evaluate_expression (expr.get ());
> +
> + if (tdb_testinfo->log_progress)
> + {
> + struct value_print_options opts;
> +
> + get_user_print_options (&opts);
> + LOG (" = ");
> + value_print (val, gdb_stdlog, &opts);
> + }
> + }
> +
> + LOG (" ... OK\n");
> +
> +#undef LOG
> +#undef __CHECK
> +#undef CHECK
> +#undef CHECK_CALL
> +
> + return 0;
> +}
> +
> +/* Run integrity checks on the dlopen()ed libthread_db described by
> + INFO. Returns true on success, displays a warning and returns
> + false on failure. Logs progress messages to gdb_stdlog during
> + the test if LOG_PROGRESS is true. */
> +
> +static bool
> +check_thread_db (struct thread_db_info *info, bool log_progress)
> +{
> + bool test_passed = true;
> +
> + if (log_progress)
> + debug_printf (_("Running libthread_db integrity checks:\n"));
> +
> + /* GDB avoids using td_ta_thr_iter wherever possible (see comment
> + in try_thread_db_load_1 below) so in order to test it we may
> + have to locate it ourselves. */
> + td_ta_thr_iter_ftype *td_ta_thr_iter_p = info->td_ta_thr_iter_p;
> + if (td_ta_thr_iter_p == NULL)
> + {
> + void *thr_iter = verbose_dlsym (info->handle, "td_ta_thr_iter");
> + if (thr_iter == NULL)
> + return 0;
> +
> + td_ta_thr_iter_p = (td_ta_thr_iter_ftype *) thr_iter;
> + }
> +
> + /* Set up the test state we share with the callback. */
> + gdb_assert (tdb_testinfo == NULL);
> + struct check_thread_db_info tdb_testinfo_buf;
> + tdb_testinfo = &tdb_testinfo_buf;
> +
> + memset (tdb_testinfo, 0, sizeof (struct check_thread_db_info));
> + tdb_testinfo->info = info;
> + tdb_testinfo->log_progress = log_progress;
Just a nit, but check_thread_db_info could use a constructor that receives
these as parameters.
> +
> + /* td_ta_thr_iter shouldn't be used on running processes. */
> + linux_stop_and_wait_all_lwps ();
> +
> + TRY
> + {
> + td_err_e err = td_ta_thr_iter_p (info->thread_agent,
> + check_thread_db_callback,
> + tdb_testinfo,
> + TD_THR_ANY_STATE,
> + TD_THR_LOWEST_PRIORITY,
> + TD_SIGNO_MASK,
> + TD_THR_ANY_USER_FLAGS);
> +
> + if (err != TD_OK)
> + error (_("td_ta_thr_iter failed: %s"), thread_db_err_str (err));
> +
> + if (!tdb_testinfo->threads_seen)
> + error (_("no threads seen"));
> + }
> + CATCH (except, RETURN_MASK_ERROR)
> + {
> + if (warning_pre_print)
> + fputs_unfiltered (warning_pre_print, gdb_stderr);
> +
> + exception_fprintf (gdb_stderr, except,
> + _("libthread_db integrity checks failed: "));
> +
> + test_passed = false;
> + }
> + END_CATCH
> +
> + if (test_passed && log_progress)
> + debug_printf (_("libthread_db integrity checks passed.\n"));
> +
> + tdb_testinfo = NULL;
> +
> + linux_unstop_all_lwps ();
> +
> + return test_passed;
> +}
> +
> /* Attempt to initialize dlopen()ed libthread_db, described by INFO.
> Return 1 on success.
> Failure could happen if libthread_db does not have symbols we expect,
> @@ -586,6 +801,13 @@ try_thread_db_load_1 (struct thread_db_info *info)
> #undef TDB_DLSYM
> #undef CHK
>
> + /* Run integrity checks if requested. */
> + if (check_thread_db_on_load)
> + {
> + if (!check_thread_db (info, libthread_db_debug != 0))
You can use simply "libthread_db_debug" instead of "libthread_db_debug != 0",
since it's semantically a bool.
> + return 0;
> + }
> +
> if (info->td_ta_thr_iter_p == NULL)
> {
> struct lwp_info *lp;
> @@ -1704,6 +1926,24 @@ info_auto_load_libthread_db (const char *args, int from_tty)
> uiout->message (_("No auto-loaded libthread-db.\n"));
> }
>
> +/* Implement 'maintenance check libthread-db'. */
> +
> +static void
> +maintenance_check_libthread_db (const char *args, int from_tty)
> +{
> + int inferior_pid = ptid_get_pid (inferior_ptid);
> + struct thread_db_info *info;
> +
> + if (inferior_pid == 0)
> + error (_("No inferior running"));
> +
> + info = get_thread_db_info (inferior_pid);
> + if (info == NULL)
> + error (_("No libthread_db loaded"));
> +
> + check_thread_db (info, true);
> +}
> +
> static void
> init_thread_db_ops (void)
> {
> @@ -1780,6 +2020,23 @@ This options has security implications for untrusted inferiors."),
> Usage: info auto-load libthread-db"),
> auto_load_info_cmdlist_get ());
>
> + add_cmd ("libthread-db", class_maintenance,
> + maintenance_check_libthread_db, _("\
> +Run integrity checks on the current inferior's libthread_db."),
> + &maintenancechecklist);
> +
> + add_setshow_boolean_cmd ("check-libthread-db",
> + class_maintenance,
> + &check_thread_db_on_load, _("\
> +Set whether to check libthread_db at load time."), _("\
> +Show whether to check libthread_db at load time."), _("\
> +If enabled GDB will run integrity checks on inferior specific libthread_db\n\
> +as they are loaded."),
> + NULL,
> + NULL,
> + &maintenance_set_cmdlist,
> + &maintenance_show_cmdlist);
> +
> /* Add ourselves to objfile event chain. */
> observer_attach_new_objfile (thread_db_new_objfile);
>
> diff --git a/gdb/testsuite/gdb.threads/check-libthread-db.c b/gdb/testsuite/gdb.threads/check-libthread-db.c
> new file mode 100644
> index 0000000..22be229
> --- /dev/null
> +++ b/gdb/testsuite/gdb.threads/check-libthread-db.c
> @@ -0,0 +1,66 @@
> +/* This testcase is part of GDB, the GNU debugger.
> +
> + Copyright 2017 Free Software Foundation, Inc.
> +
> + This program is free software; you can redistribute it and/or modify
> + it under the terms of the GNU General Public License as published by
> + the Free Software Foundation; either version 3 of the License, or
> + (at your option) any later version.
> +
> + This program is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + GNU General Public License for more details.
> +
> + You should have received a copy of the GNU General Public License
> + along with this program. If not, see <http://www.gnu.org/licenses/>. */
> +
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <sys/types.h>
> +#include <unistd.h>
> +#include <pthread.h>
> +#include <errno.h>
> +
> +static void
> +break_here (void)
> +{
> +}
> +
> +static void *
> +thread_routine (void *arg)
> +{
> + errno = 42;
> +
> + break_here ();
> +
> + while (1)
> + sleep (1);
> +
> + return NULL;
> +}
> +
> +int
> +main (int argc, char *argv)
> +{
> + pthread_t the_thread;
> + int err;
> +
> + err = pthread_create (&the_thread, NULL, thread_routine, NULL);
> + if (err != 0)
> + {
> + fprintf (stderr, "pthread_create: %s (%d)\n", strerror (err), err);
> + exit (EXIT_FAILURE);
> + }
> +
> + errno = 23;
> +
> + err = pthread_join (the_thread, NULL);
> + if (err != 0)
> + {
> + fprintf (stderr, "pthread_join: %s (%d)\n", strerror (err), err);
> + exit (EXIT_FAILURE);
> + }
> +
> + exit (EXIT_SUCCESS);
> +}
> diff --git a/gdb/testsuite/gdb.threads/check-libthread-db.exp b/gdb/testsuite/gdb.threads/check-libthread-db.exp
> new file mode 100644
> index 0000000..3990cf3
> --- /dev/null
> +++ b/gdb/testsuite/gdb.threads/check-libthread-db.exp
> @@ -0,0 +1,114 @@
> +# Copyright 2017 Free Software Foundation, Inc.
> +
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 3 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program. If not, see <http://www.gnu.org/licenses/>.
> +
> +# This test only works for native processes on Linux.
> +if { ![isnative] || [is_remote host] || [target_info exists use_gdb_stub]
> + || ![istarget *-linux*] } {
> + continue
> +}
> +
> +standard_testfile
> +
> +if {[gdb_compile_pthreads "${srcdir}/${subdir}/${srcfile}" "${binfile}" \
> + executable debug] != "" } {
> + return -1
> +}
> +
> +# TEST 1: Manual check with libthread_db not loaded.
> +
> +clean_restart ${binfile}
> +
> +gdb_test "maint show check-libthread-db" \
> + "Whether to check libthread_db at load time is off."
> +
> +gdb_test_no_output "set stop-on-solib-events 1"
> +gdb_run_cmd
> +gdb_test "" \
> + ".*Stopped due to shared library event.*no libraries added or removed.*"
> +
> +gdb_test "maint check libthread-db" \
> + "No libthread_db loaded" \
> + "libpthread.so not loaded"
> +
> +
> +# TEST 2: Manual check with NPTL uninitialized.
> +# libthread_db should fake a single thread with th_unique == NULL.
> +
> +gdb_test "continue" \
> + ".*Stopped due to shared library event.*Inferior loaded .*libpthread.*"
> +
> +gdb_test_sequence "maint check libthread-db" \
> + "libpthread.so not initialized (manual)" {
Don't use test names that end with something in parentheses, see:
https://sourceware.org/gdb/wiki/GDBTestcaseCookbook#Do_not_use_.22tail_parentheses.22_on_test_messages
To keep names unique, you can either just change the format of the message,
or wrap the code with
with_test_prefix "manual" {
...
}
with_test_prefix "automated" {
...
}
> + "\[\r\n\]+Running libthread_db integrity checks:"
> + "\[\r\n\]+\[ \]+Got thread 0x0 => \[0-9\]+ => 0x0 ... OK"
> + "\[\r\n\]+libthread_db integrity checks passed."
> + }
> +
> +
> +# TEST 3: Manual check with NPTL fully operational.
> +
> +gdb_test_no_output "set stop-on-solib-events 0"
> +gdb_breakpoint break_here
> +gdb_continue_to_breakpoint break_here
> +
> +gdb_test_sequence "maint check libthread-db" \
> + "libpthread.so fully initialized (manual)" {
> + "\[\r\n\]+Running libthread_db integrity checks:"
> + "\[\r\n\]+\[ \]+Got thread 0x\[1-9a-f\]\[0-9a-f\]+ => \[0-9\]+ => 0x\[1-9a-f\]\[0-9a-f\]+; errno = 23 ... OK"
> + "\[\r\n\]+\[ \]+Got thread 0x\[1-9a-f\]\[0-9a-f\]+ => \[0-9\]+ => 0x\[1-9a-f\]\[0-9a-f\]+; errno = 42 ... OK"
> + "\[\r\n\]+libthread_db integrity checks passed."
> + }
> +
> +
> +# TEST 4: Automated check with NPTL uninitialized.
> +
> +clean_restart ${binfile}
> +
> +gdb_test_no_output "maint set check-libthread-db 1"
> +gdb_test_no_output "set debug libthread-db 1"
> +gdb_breakpoint break_here
> +gdb_run_cmd
> +
> +gdb_test_sequence "" \
> + "libpthread.so not initialized (automated)" {
> + "\[\r\n\]+Running libthread_db integrity checks:"
> + "\[\r\n\]+\[ \]+Got thread 0x0 => \[0-9\]+ => 0x0 ... OK"
> + "\[\r\n\]+libthread_db integrity checks passed."
> + "\[\r\n\]+[Thread debugging using libthread_db enabled]"
> + }
> +
> +
> +# TEST 5: Automated check with NPTL fully operational.
> +
> +clean_restart ${binfile}
> +
> +gdb_test_no_output "maint set check-libthread-db 1"
> +gdb_test_no_output "set debug libthread-db 1"
> +
> +set test_spawn_id [spawn_wait_for_attach $binfile]
> +set testpid [spawn_id_get_pid $test_spawn_id]
> +
> +gdb_test_sequence "attach $testpid" \
> + "libpthread.so fully initialized (automated)" {
> + "\[\r\n\]+Running libthread_db integrity checks:"
> + "\[\r\n\]+\[ \]+Got thread 0x\[1-9a-f\]\[0-9a-f\]+ => \[0-9\]+ => 0x\[1-9a-f\]\[0-9a-f\]+ ... OK"
> + "\[\r\n\]+\[ \]+Got thread 0x\[1-9a-f\]\[0-9a-f\]+ => \[0-9\]+ => 0x\[1-9a-f\]\[0-9a-f\]+ ... OK"
> + "\[\r\n\]+libthread_db integrity checks passed."
> + "\[\r\n\]+[Thread debugging using libthread_db enabled]"
> + }
> +
> +
> +gdb_exit
> +kill_wait_spawned_process $test_spawn_id
>
Thanks,
Simon
On 11/22/2017 02:42 PM, Gary Benson wrote:
>
> Built and regtested on RHEL 7.4 x86_64.
>
> Ok to commit?
I noticed you're missing a NEWS entry.
Some more comments below.
> +static int
> +check_thread_db_callback (const td_thrhandle_t *th, void *arg)
> +{
> + gdb_assert (tdb_testinfo != NULL);
> + tdb_testinfo->threads_seen = true;
> +
> +#define LOG(fmt, args...) \
> + do \
> + { \
> + if (tdb_testinfo->log_progress) \
> + { \
> + debug_printf (fmt, ## args); \
> + gdb_flush (gdb_stdlog); \
> + } \
> + } \
> + while (0)
> +
> +#define __CHECK(expr, args...) \
Symbols that start with __ are reserved for the C/C++ implementation.
(E.g., glibc would be in the right to define it in some header
for its own purposes).
You can rename it to CHECK_1 or something like that.
> + do \
> + { \
> + if (!(expr)) \
> + { \
> + LOG (" ... FAIL!\n"); \
> + error (args); \
> + } \
> + } \
> + while (0)
> +
> +#define CHECK(expr) \
> + __CHECK (expr, "(%s) == false", #expr)
> +
> +#define CHECK_CALL(func, args...) \
> + do \
> + { \
> + td_err_e __err = tdb_testinfo->info->func ## _p (args); \
> + \
> + __CHECK (__err == TD_OK, _("%s failed: %s"), #func, \
> + thread_db_err_str (__err)); \
Likewise __err.
> +# You should have received a copy of the GNU General Public License
> +# along with this program. If not, see <http://www.gnu.org/licenses/>.
> +
> +# This test only works for native processes on Linux.
GNU/Linux, because glibc.
> +if { ![isnative] || [is_remote host] || [target_info exists use_gdb_stub]
> + || ![istarget *-linux*] } {
> + continue
I think this fails to filter out --target_board=native-extended-gdbserver ?
This would be more to the point, and would work, I think (haven't tested):
# This test only works for native processes on GNU/Linux.
if {[target_info gdb_protocol] != "" || ![istarget *-linux*]} {
continue
}
> +}
> +
> +standard_testfile
> +
> +if {[gdb_compile_pthreads "${srcdir}/${subdir}/${srcfile}" "${binfile}" \
> + executable debug] != "" } {
> + return -1
> +}
> +
> +# TEST 1: Manual check with libthread_db not loaded.
Please remove the numbering in "TEST 1", etc. We have a few old
testcases that have comments like that, and over the years as tests
are added/removed to the file, the numbering gets annoyingly in the
way / odd, when inevitably in the future we want to insert a
test in the middle, remove or reorder tests, etc.
> +
> +clean_restart ${binfile}
> +
> +gdb_test "maint show check-libthread-db" \
> + "Whether to check libthread_db at load time is off."
> +
> +gdb_test_no_output "set stop-on-solib-events 1"
> +gdb_run_cmd
> +gdb_test "" \
> + ".*Stopped due to shared library event.*no libraries added or removed.*"
> +
> +gdb_test "maint check libthread-db" \
> + "No libthread_db loaded" \
> + "libpthread.so not loaded"
> +
> +
> +# TEST 2: Manual check with NPTL uninitialized.
> +# libthread_db should fake a single thread with th_unique == NULL.
> +
> +gdb_test "continue" \
> + ".*Stopped due to shared library event.*Inferior loaded .*libpthread.*"
> +
> +gdb_test_sequence "maint check libthread-db" \
> + "libpthread.so not initialized (manual)" {
> + "\[\r\n\]+Running libthread_db integrity checks:"
> + "\[\r\n\]+\[ \]+Got thread 0x0 => \[0-9\]+ => 0x0 ... OK"
> + "\[\r\n\]+libthread_db integrity checks passed."
> + }
> +
> +
> +# TEST 3: Manual check with NPTL fully operational.
> +
> +gdb_test_no_output "set stop-on-solib-events 0"
> +gdb_breakpoint break_here
> +gdb_continue_to_breakpoint break_here
> +
> +gdb_test_sequence "maint check libthread-db" \
> + "libpthread.so fully initialized (manual)" {
> + "\[\r\n\]+Running libthread_db integrity checks:"
> + "\[\r\n\]+\[ \]+Got thread 0x\[1-9a-f\]\[0-9a-f\]+ => \[0-9\]+ => 0x\[1-9a-f\]\[0-9a-f\]+; errno = 23 ... OK"
> + "\[\r\n\]+\[ \]+Got thread 0x\[1-9a-f\]\[0-9a-f\]+ => \[0-9\]+ => 0x\[1-9a-f\]\[0-9a-f\]+; errno = 42 ... OK"
> + "\[\r\n\]+libthread_db integrity checks passed."
> + }
> +
> +
> +# TEST 4: Automated check with NPTL uninitialized.
> +
> +clean_restart ${binfile}
> +
> +gdb_test_no_output "maint set check-libthread-db 1"
> +gdb_test_no_output "set debug libthread-db 1"
> +gdb_breakpoint break_here
> +gdb_run_cmd
> +
> +gdb_test_sequence "" \
> + "libpthread.so not initialized (automated)" {
> + "\[\r\n\]+Running libthread_db integrity checks:"
> + "\[\r\n\]+\[ \]+Got thread 0x0 => \[0-9\]+ => 0x0 ... OK"
> + "\[\r\n\]+libthread_db integrity checks passed."
> + "\[\r\n\]+[Thread debugging using libthread_db enabled]"
> + }
> +
> +
> +# TEST 5: Automated check with NPTL fully operational.
> +
> +clean_restart ${binfile}
> +
> +gdb_test_no_output "maint set check-libthread-db 1"
> +gdb_test_no_output "set debug libthread-db 1"
> +
> +set test_spawn_id [spawn_wait_for_attach $binfile]
> +set testpid [spawn_id_get_pid $test_spawn_id]
> +
> +gdb_test_sequence "attach $testpid" \
> + "libpthread.so fully initialized (automated)" {
> + "\[\r\n\]+Running libthread_db integrity checks:"
> + "\[\r\n\]+\[ \]+Got thread 0x\[1-9a-f\]\[0-9a-f\]+ => \[0-9\]+ => 0x\[1-9a-f\]\[0-9a-f\]+ ... OK"
> + "\[\r\n\]+\[ \]+Got thread 0x\[1-9a-f\]\[0-9a-f\]+ => \[0-9\]+ => 0x\[1-9a-f\]\[0-9a-f\]+ ... OK"
> + "\[\r\n\]+libthread_db integrity checks passed."
> + "\[\r\n\]+[Thread debugging using libthread_db enabled]"
> + }
> +
> +
> +gdb_exit
> +kill_wait_spawned_process $test_spawn_id
>
I'm surprised to not see any validation when debugging
a core dump. Would it work?
Thanks,
Pedro Alves
@@ -34981,6 +34981,11 @@ modify XML target descriptions.
Check that the target descriptions dynamically created by @value{GDBN}
equal the descriptions created from XML files found in @var{dir}.
+@kindex maint check libthread-db
+@item maint check libthread-db
+Run integrity checks on the current inferior's thread debugging
+library.
+
@kindex maint print dummy-frames
@item maint print dummy-frames
Prints the contents of @value{GDBN}'s internal dummy-frame stack.
@@ -35288,6 +35293,14 @@ number of blocks in the blockvector
@end enumerate
@end table
+@kindex maint set check-libthread-db
+@kindex maint show check-libthread-db
+@item maint set check-libthread-db [on|off]
+@itemx maint show check-libthread-db
+Control whether @value{GDBN} should run integrity checks on inferior
+specific thread debugging libraries as they are loaded. The default
+is not to perform such checks.
+
@kindex maint space
@cindex memory used by commands
@item maint space @var{value}
@@ -46,6 +46,7 @@
#include <ctype.h>
#include "nat/linux-namespaces.h"
#include <algorithm>
+#include "valprint.h"
/* GNU/Linux libthread_db support.
@@ -80,6 +81,10 @@ static char *libthread_db_search_path;
by the "set auto-load libthread-db" command. */
static int auto_load_thread_db = 1;
+/* Set to non-zero if load-time libthread_db tests have been enabled
+ by the "maintenence set check-libthread-db" command. */
+static int check_thread_db_on_load = 0;
+
/* "show" command for the auto_load_thread_db configuration variable. */
static void
@@ -493,6 +498,216 @@ dladdr_to_soname (const void *addr)
return NULL;
}
+/* State for check_thread_db_callback. */
+
+struct check_thread_db_info
+{
+ /* The libthread_db under test. */
+ struct thread_db_info *info;
+
+ /* True if progress should be logged. */
+ bool log_progress;
+
+ /* True if the callback was called. */
+ bool threads_seen;
+};
+
+static struct check_thread_db_info *tdb_testinfo;
+
+/* Callback for check_thread_db. */
+
+static int
+check_thread_db_callback (const td_thrhandle_t *th, void *arg)
+{
+ gdb_assert (tdb_testinfo != NULL);
+ tdb_testinfo->threads_seen = true;
+
+#define LOG(fmt, args...) \
+ do \
+ { \
+ if (tdb_testinfo->log_progress) \
+ { \
+ debug_printf (fmt, ## args); \
+ gdb_flush (gdb_stdlog); \
+ } \
+ } \
+ while (0)
+
+#define __CHECK(expr, args...) \
+ do \
+ { \
+ if (!(expr)) \
+ { \
+ LOG (" ... FAIL!\n"); \
+ error (args); \
+ } \
+ } \
+ while (0)
+
+#define CHECK(expr) \
+ __CHECK (expr, "(%s) == false", #expr)
+
+#define CHECK_CALL(func, args...) \
+ do \
+ { \
+ td_err_e __err = tdb_testinfo->info->func ## _p (args); \
+ \
+ __CHECK (__err == TD_OK, _("%s failed: %s"), #func, \
+ thread_db_err_str (__err)); \
+ } \
+ while (0)
+
+ LOG (" Got thread");
+
+ /* Check td_ta_thr_iter passed consistent arguments. */
+ CHECK (th != NULL);
+ CHECK (arg == (void *) tdb_testinfo);
+ CHECK (th->th_ta_p == tdb_testinfo->info->thread_agent);
+
+ LOG (" %s", core_addr_to_string_nz ((CORE_ADDR) th->th_unique));
+
+ /* Check td_thr_get_info. */
+ td_thrinfo_t ti;
+ CHECK_CALL (td_thr_get_info, th, &ti);
+
+ LOG (" => %d", ti.ti_lid);
+
+ CHECK (ti.ti_ta_p == th->th_ta_p);
+ CHECK (ti.ti_tid == (thread_t) th->th_unique);
+
+ /* Check td_ta_map_lwp2thr. */
+ td_thrhandle_t th2;
+ memset (&th2, 23, sizeof (td_thrhandle_t));
+ CHECK_CALL (td_ta_map_lwp2thr, th->th_ta_p, ti.ti_lid, &th2);
+
+ LOG (" => %s", core_addr_to_string_nz ((CORE_ADDR) th2.th_unique));
+
+ CHECK (memcmp (th, &th2, sizeof (td_thrhandle_t)) == 0);
+
+ /* Attempt TLS access. Assuming errno is TLS, this calls
+ thread_db_get_thread_local_address, which in turn calls
+ td_thr_tls_get_addr for live inferiors or td_thr_tlsbase
+ for core files. This test is skipped if the thread has
+ not been recorded; proceeding in that case would result
+ in the test having the side-effect of noticing threads
+ which seems wrong.
+
+ Note that in glibc's libthread_db td_thr_tls_get_addr is
+ a thin wrapper around td_thr_tlsbase; this check always
+ hits the bulk of the code.
+
+ Note also that we don't actually check any libthread_db
+ calls are made, we just assume they were; future changes
+ to how GDB accesses TLS could result in this passing
+ without exercising the calls it's supposed to. */
+ ptid_t ptid = ptid_build (tdb_testinfo->info->pid, ti.ti_lid, 0);
+ struct thread_info *thread_info = find_thread_ptid (ptid);
+ if (thread_info != NULL && thread_info->priv != NULL)
+ {
+ LOG ("; errno");
+
+ scoped_restore_current_thread restore_current_thread;
+ switch_to_thread (ptid);
+
+ expression_up expr = parse_expression ("errno");
+ struct value *val = evaluate_expression (expr.get ());
+
+ if (tdb_testinfo->log_progress)
+ {
+ struct value_print_options opts;
+
+ get_user_print_options (&opts);
+ LOG (" = ");
+ value_print (val, gdb_stdlog, &opts);
+ }
+ }
+
+ LOG (" ... OK\n");
+
+#undef LOG
+#undef __CHECK
+#undef CHECK
+#undef CHECK_CALL
+
+ return 0;
+}
+
+/* Run integrity checks on the dlopen()ed libthread_db described by
+ INFO. Returns true on success, displays a warning and returns
+ false on failure. Logs progress messages to gdb_stdlog during
+ the test if LOG_PROGRESS is true. */
+
+static bool
+check_thread_db (struct thread_db_info *info, bool log_progress)
+{
+ bool test_passed = true;
+
+ if (log_progress)
+ debug_printf (_("Running libthread_db integrity checks:\n"));
+
+ /* GDB avoids using td_ta_thr_iter wherever possible (see comment
+ in try_thread_db_load_1 below) so in order to test it we may
+ have to locate it ourselves. */
+ td_ta_thr_iter_ftype *td_ta_thr_iter_p = info->td_ta_thr_iter_p;
+ if (td_ta_thr_iter_p == NULL)
+ {
+ void *thr_iter = verbose_dlsym (info->handle, "td_ta_thr_iter");
+ if (thr_iter == NULL)
+ return 0;
+
+ td_ta_thr_iter_p = (td_ta_thr_iter_ftype *) thr_iter;
+ }
+
+ /* Set up the test state we share with the callback. */
+ gdb_assert (tdb_testinfo == NULL);
+ struct check_thread_db_info tdb_testinfo_buf;
+ tdb_testinfo = &tdb_testinfo_buf;
+
+ memset (tdb_testinfo, 0, sizeof (struct check_thread_db_info));
+ tdb_testinfo->info = info;
+ tdb_testinfo->log_progress = log_progress;
+
+ /* td_ta_thr_iter shouldn't be used on running processes. */
+ linux_stop_and_wait_all_lwps ();
+
+ TRY
+ {
+ td_err_e err = td_ta_thr_iter_p (info->thread_agent,
+ check_thread_db_callback,
+ tdb_testinfo,
+ TD_THR_ANY_STATE,
+ TD_THR_LOWEST_PRIORITY,
+ TD_SIGNO_MASK,
+ TD_THR_ANY_USER_FLAGS);
+
+ if (err != TD_OK)
+ error (_("td_ta_thr_iter failed: %s"), thread_db_err_str (err));
+
+ if (!tdb_testinfo->threads_seen)
+ error (_("no threads seen"));
+ }
+ CATCH (except, RETURN_MASK_ERROR)
+ {
+ if (warning_pre_print)
+ fputs_unfiltered (warning_pre_print, gdb_stderr);
+
+ exception_fprintf (gdb_stderr, except,
+ _("libthread_db integrity checks failed: "));
+
+ test_passed = false;
+ }
+ END_CATCH
+
+ if (test_passed && log_progress)
+ debug_printf (_("libthread_db integrity checks passed.\n"));
+
+ tdb_testinfo = NULL;
+
+ linux_unstop_all_lwps ();
+
+ return test_passed;
+}
+
/* Attempt to initialize dlopen()ed libthread_db, described by INFO.
Return 1 on success.
Failure could happen if libthread_db does not have symbols we expect,
@@ -586,6 +801,13 @@ try_thread_db_load_1 (struct thread_db_info *info)
#undef TDB_DLSYM
#undef CHK
+ /* Run integrity checks if requested. */
+ if (check_thread_db_on_load)
+ {
+ if (!check_thread_db (info, libthread_db_debug != 0))
+ return 0;
+ }
+
if (info->td_ta_thr_iter_p == NULL)
{
struct lwp_info *lp;
@@ -1704,6 +1926,24 @@ info_auto_load_libthread_db (const char *args, int from_tty)
uiout->message (_("No auto-loaded libthread-db.\n"));
}
+/* Implement 'maintenance check libthread-db'. */
+
+static void
+maintenance_check_libthread_db (const char *args, int from_tty)
+{
+ int inferior_pid = ptid_get_pid (inferior_ptid);
+ struct thread_db_info *info;
+
+ if (inferior_pid == 0)
+ error (_("No inferior running"));
+
+ info = get_thread_db_info (inferior_pid);
+ if (info == NULL)
+ error (_("No libthread_db loaded"));
+
+ check_thread_db (info, true);
+}
+
static void
init_thread_db_ops (void)
{
@@ -1780,6 +2020,23 @@ This options has security implications for untrusted inferiors."),
Usage: info auto-load libthread-db"),
auto_load_info_cmdlist_get ());
+ add_cmd ("libthread-db", class_maintenance,
+ maintenance_check_libthread_db, _("\
+Run integrity checks on the current inferior's libthread_db."),
+ &maintenancechecklist);
+
+ add_setshow_boolean_cmd ("check-libthread-db",
+ class_maintenance,
+ &check_thread_db_on_load, _("\
+Set whether to check libthread_db at load time."), _("\
+Show whether to check libthread_db at load time."), _("\
+If enabled GDB will run integrity checks on inferior specific libthread_db\n\
+as they are loaded."),
+ NULL,
+ NULL,
+ &maintenance_set_cmdlist,
+ &maintenance_show_cmdlist);
+
/* Add ourselves to objfile event chain. */
observer_attach_new_objfile (thread_db_new_objfile);
new file mode 100644
@@ -0,0 +1,66 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+ Copyright 2017 Free Software Foundation, Inc.
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/types.h>
+#include <unistd.h>
+#include <pthread.h>
+#include <errno.h>
+
+static void
+break_here (void)
+{
+}
+
+static void *
+thread_routine (void *arg)
+{
+ errno = 42;
+
+ break_here ();
+
+ while (1)
+ sleep (1);
+
+ return NULL;
+}
+
+int
+main (int argc, char *argv)
+{
+ pthread_t the_thread;
+ int err;
+
+ err = pthread_create (&the_thread, NULL, thread_routine, NULL);
+ if (err != 0)
+ {
+ fprintf (stderr, "pthread_create: %s (%d)\n", strerror (err), err);
+ exit (EXIT_FAILURE);
+ }
+
+ errno = 23;
+
+ err = pthread_join (the_thread, NULL);
+ if (err != 0)
+ {
+ fprintf (stderr, "pthread_join: %s (%d)\n", strerror (err), err);
+ exit (EXIT_FAILURE);
+ }
+
+ exit (EXIT_SUCCESS);
+}
new file mode 100644
@@ -0,0 +1,114 @@
+# Copyright 2017 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# This test only works for native processes on Linux.
+if { ![isnative] || [is_remote host] || [target_info exists use_gdb_stub]
+ || ![istarget *-linux*] } {
+ continue
+}
+
+standard_testfile
+
+if {[gdb_compile_pthreads "${srcdir}/${subdir}/${srcfile}" "${binfile}" \
+ executable debug] != "" } {
+ return -1
+}
+
+# TEST 1: Manual check with libthread_db not loaded.
+
+clean_restart ${binfile}
+
+gdb_test "maint show check-libthread-db" \
+ "Whether to check libthread_db at load time is off."
+
+gdb_test_no_output "set stop-on-solib-events 1"
+gdb_run_cmd
+gdb_test "" \
+ ".*Stopped due to shared library event.*no libraries added or removed.*"
+
+gdb_test "maint check libthread-db" \
+ "No libthread_db loaded" \
+ "libpthread.so not loaded"
+
+
+# TEST 2: Manual check with NPTL uninitialized.
+# libthread_db should fake a single thread with th_unique == NULL.
+
+gdb_test "continue" \
+ ".*Stopped due to shared library event.*Inferior loaded .*libpthread.*"
+
+gdb_test_sequence "maint check libthread-db" \
+ "libpthread.so not initialized (manual)" {
+ "\[\r\n\]+Running libthread_db integrity checks:"
+ "\[\r\n\]+\[ \]+Got thread 0x0 => \[0-9\]+ => 0x0 ... OK"
+ "\[\r\n\]+libthread_db integrity checks passed."
+ }
+
+
+# TEST 3: Manual check with NPTL fully operational.
+
+gdb_test_no_output "set stop-on-solib-events 0"
+gdb_breakpoint break_here
+gdb_continue_to_breakpoint break_here
+
+gdb_test_sequence "maint check libthread-db" \
+ "libpthread.so fully initialized (manual)" {
+ "\[\r\n\]+Running libthread_db integrity checks:"
+ "\[\r\n\]+\[ \]+Got thread 0x\[1-9a-f\]\[0-9a-f\]+ => \[0-9\]+ => 0x\[1-9a-f\]\[0-9a-f\]+; errno = 23 ... OK"
+ "\[\r\n\]+\[ \]+Got thread 0x\[1-9a-f\]\[0-9a-f\]+ => \[0-9\]+ => 0x\[1-9a-f\]\[0-9a-f\]+; errno = 42 ... OK"
+ "\[\r\n\]+libthread_db integrity checks passed."
+ }
+
+
+# TEST 4: Automated check with NPTL uninitialized.
+
+clean_restart ${binfile}
+
+gdb_test_no_output "maint set check-libthread-db 1"
+gdb_test_no_output "set debug libthread-db 1"
+gdb_breakpoint break_here
+gdb_run_cmd
+
+gdb_test_sequence "" \
+ "libpthread.so not initialized (automated)" {
+ "\[\r\n\]+Running libthread_db integrity checks:"
+ "\[\r\n\]+\[ \]+Got thread 0x0 => \[0-9\]+ => 0x0 ... OK"
+ "\[\r\n\]+libthread_db integrity checks passed."
+ "\[\r\n\]+[Thread debugging using libthread_db enabled]"
+ }
+
+
+# TEST 5: Automated check with NPTL fully operational.
+
+clean_restart ${binfile}
+
+gdb_test_no_output "maint set check-libthread-db 1"
+gdb_test_no_output "set debug libthread-db 1"
+
+set test_spawn_id [spawn_wait_for_attach $binfile]
+set testpid [spawn_id_get_pid $test_spawn_id]
+
+gdb_test_sequence "attach $testpid" \
+ "libpthread.so fully initialized (automated)" {
+ "\[\r\n\]+Running libthread_db integrity checks:"
+ "\[\r\n\]+\[ \]+Got thread 0x\[1-9a-f\]\[0-9a-f\]+ => \[0-9\]+ => 0x\[1-9a-f\]\[0-9a-f\]+ ... OK"
+ "\[\r\n\]+\[ \]+Got thread 0x\[1-9a-f\]\[0-9a-f\]+ => \[0-9\]+ => 0x\[1-9a-f\]\[0-9a-f\]+ ... OK"
+ "\[\r\n\]+libthread_db integrity checks passed."
+ "\[\r\n\]+[Thread debugging using libthread_db enabled]"
+ }
+
+
+gdb_exit
+kill_wait_spawned_process $test_spawn_id