[1/5] Extended-remote exec events

This patch is the latest version implementing support for exec
events on extended-remote Linux targets.  Follow-exec-mode and
rerun behave as expected.  Catchpoints and test updates are
implemented in subsequent patches.

This patch was most recently derived from a patch posted last
October: https://sourceware.org/ml/gdb-patches/2014-10/msg00877.html.
It was originally based on some work done by Luis Machado in 2013.

IMPLEMENTATION
----------------
Support for exec events in single-threaded programs was a fairly
straightforward replication of the implementation in native GDB:

1) Enable exec events via ptrace options.

2) Add support for handling the exec events to the handle_extended_wait and
linux_wait_for_event_filtered.  Detect the exec event, then find and save
the pathname of the executable file being exec'd and set event status flags.

3) Implement an additional "stop reason", "exec", in the RSP stop reply
packet "T".

Existing GDB code takes care of handling the exec event on the host side
without modification.

Support for exec events in multi-threaded programs required some additional
work.  When exec is called, the Linux kernel destroys all of the threads
except the execing one.  If the execing thread was not the thread group
leader, the kernel resets the execing thread's tid to the tgid, and no
exit notification is sent for the execing thread -- from the ptracer's
perspective, it appears as though the execing thread just vanishes.

The non-leader exec leaves gdbserver with one or more of several
potential scenarios which require it to bring its thread lists (e.g.
struct thread_info, struct lwp_info) in sync with reality.

 - The leader thread exited before the exec event was reported, and
   the execing thread cannot re-use its data structures.

   In this case gdbserver must recognize that an exec event occurred
   and there are no thread structures for the leader thread, so it
   must add new structures to the lists for the 'new' leader thread.

  if (WIFSTOPPED (wstat) && (child == NULL) && (WSTOPSIG (wstat) == SIGTRAP)
      && (linux_ptrace_get_extended_event (wstat) == PTRACE_EVENT_EXEC))
    {
      ptid_t child_ptid;

      /* A multi-thread exec after we had seen the leader exiting.  */
      if (debug_threads)
	{
	  debug_printf ("LLW: Re-adding thread group leader LWP %d"
			"after exec.\n", lwpid);
	}

      child_ptid = ptid_build (lwpid, lwpid, 0);
      child = add_lwp (child_ptid);
      child->stopped = 1;
      current_thread = child->thread;
    }

 - The execing thread can re-use the previous leader thread's data
   structures, and the old data structures used for the execing thread
   prior to the exec are left with a running status and no actual
   thread associated with it.

   When a non-leader execing thread re-uses the previous leader's
   thread_info structure, it inherits the old thread's register
   cache.  If this is left as-is it will eventually be flushed to
   the target, clobbering the valid register values with those from
   the old thread.  So when an EXEC event occurs we always invalidate
   the register cache.  Note that we can't call regcache_invalidate,
   since that flushes the cache to the target.

  else if (event == PTRACE_EVENT_EXEC && report_exec_events)
    {
      struct regcache *regcache;
      ---snip snip---
      regcache = (struct regcache *) inferior_regcache_data (event_thr);
      free_register_cache (regcache);
      set_inferior_regcache_data (event_thr, NULL);
      event_lwp->stop_pc = get_pc (event_lwp);

   In this case (stale thread data structures for execing thread) when
   in all-stop mode, gdbserver must clean up any stale thread/lwp structures
   before it tries to stop all the threads and hangs in sigsuspend, waiting
   for an event from a non-existent thread.  We do this by checking the
   return value from kill_lwp in send_sigstop, eventually called after
   calling stop_all_lwps, which is used to stop the threads in all-stop
   mode.  If kill_lwp returns an error and errno is ESRCH, we know that the
   lwp with that pid is gone, and we delete the associated data structures.

-  kill_lwp (pid, SIGSTOP);
+  errno = 0;
+  ret = kill_lwp (pid, SIGSTOP);
+  if (ret == -1 && errno == ESRCH)
+    {
+      /* If the kill fails with "No such process", on GNU/Linux we know
+	 that the LWP has vanished - it is not a zombie, it is gone.
+	 This is because a thread that was not the thread group leader
+	 called exec and took over the leader's lwp.  */
+      delete_lwp (lwp);
+      set_desired_thread (0);

   In the same case in non-stop mode, we don't need to stop all the
   lwps, but in order to utilize the same mechanism used in all-stop
   mode, we call stop_all_lwps/unstop_all_lwps in succession, just
   to check for ESRCH errors and to delete any stale thread structures.

      if (non_stop && stopping_threads == NOT_STOPPING_THREADS)
	{
	  /* In non-stop mode, make sure we delete the lwp entry for a
	     non-leader exec'ing thread, which will have vanished.  We
	     do this by sending a signal to all the other threads in the
	     lwp list, deleting any that are not found.  Note that in
	     all-stop mode this will happen when we stop all the threads.  */
	  stop_all_lwps (0, event_lwp);
	  unstop_all_lwps (0, event_lwp);
	}

Note that the native implementation uses a different mechanism for
identifying the stale data structure scenario.  It determines that the
execing thread has "vanished" by calling waitpid(PID) and checking for a
return value of ECHILD, which means that the thread is gone.  We don't want
to use waitpid(PID) in gdbserver, based on the discussion in:

  https://www.sourceware.org/ml/gdb-patches/2014-02/msg00828.html

so we use the send_sigstop method described above instead.

TESTING
--------
x86_64 GNU/Linux for native, native-gdbserver, and
native-extended-gdbserver targets.  Most of the exec-related tests fail
due to the lack of catchpoints and extended-remote support in the tests.

Thanks
--Don

gdb/gdbserver/
2015-07-15  Don Breazeal  <donb@codesourcery.com>
	    Luis Machado  <lgustavo@codesourcery.com>

	* linux-low.c (handle_extended_wait): Handle exec events.
	(check_zombie_leaders): Do not check stopped threads.
	(linux_low_ptrace_options): Add PTRACE_O_TRACEEXEC.
	(linux_low_filter_event): Add thread structures for exec'ing
	non-leader thread after leader thread had been deleted.
	(linux_wait_for_event_filtered): Fix comment saying exec events
	are not supported in remote.
	(extended_event_reported): Add TARGET_WAITKIND_EXECD.
	(linux_wait_1): Prevent clobbering extended event status.
	(send_sigstop): Check return from kill_lwp, and if ESRCH then
	call delete_lwp.
	(linux_supports_exec_events): New function.
	* lynx-low.c (lynx_target_ops): Initialize new structure
	member 'supports_exec_events'.
	* remote-utils.c (prepare_resume_reply): New stop reason 'exec'.
	* server.c (report_exec_events): New global variable.
	(handle_query): Handle qSupported query for exec-events feature.
	(captured_main): Initialize report_exec_events.
	* server.h (report_exec_events): Declare new global variable.
	* target.h (struct target_ops) <supports_exec_events>: New
	member.
        (target_supports_exec_events): New macro.
	* win32-low.c (win32_target_ops): Initialize new structure
	member 'supports_exec_events'.

gdb/
2015-07-15  Don Breazeal  <donb@codesourcery.com>
	    Luis Machado  <lgustavo@codesourcery.com>

	* nat/linux-ptrace.c (linux_supports_tracefork): Delete
	out-of-date comment verbiage.
	(linux_supports_traceexec): New function.
	* nat/linux-ptrace.h (linux_supports_traceexec): Declare.
	* remote.c (anonymous enum) <PACKET_exec_event_feature> New
	enumeration constant.
	(remote_protocol_features): Add entry for exec-events feature.
	(remote_query_supported): Add client side of qSupported query
	for exec-events feature.
	(_initialize_remote): Call add_packet_config_cmd for remote
	exec-events feature.

---
 gdb/gdbserver/linux-low.c    | 117 ++++++++++++++++++++++++++++++++++++++++---
 gdb/gdbserver/lynx-low.c     |   1 +
 gdb/gdbserver/remote-utils.c |  20 ++++++++
 gdb/gdbserver/server.c       |  11 ++++
 gdb/gdbserver/server.h       |   1 +
 gdb/gdbserver/target.h       |   7 +++
 gdb/gdbserver/win32-low.c    |   1 +
 gdb/nat/linux-ptrace.c       |  13 +++--
 gdb/nat/linux-ptrace.h       |   1 +
 gdb/remote.c                 |  30 +++++++++++
 10 files changed, 193 insertions(+), 9 deletions(-)

Don Breazeal <donb@codesourcery.com> writes:

> IMPLEMENTATION
> ----------------
> Support for exec events in single-threaded programs was a fairly
> straightforward replication of the implementation in native GDB:
>
> 1) Enable exec events via ptrace options.
>
> 2) Add support for handling the exec events to the handle_extended_wait and
> linux_wait_for_event_filtered.  Detect the exec event, then find and save
> the pathname of the executable file being exec'd and set event status flags.
>
> 3) Implement an additional "stop reason", "exec", in the RSP stop reply
> packet "T".
>
> Existing GDB code takes care of handling the exec event on the host side
> without modification.

Hi Don,
How does GDBserver handle the multi-arch case? say, 64-bit process call
exec to a 32-bit program.  At least, the target description of that
process in GDBserver should be updated.

On 7/16/2015 7:00 AM, Yao Qi wrote:
> Don Breazeal <donb@codesourcery.com> writes:
> 
>> IMPLEMENTATION
>> ----------------
>> Support for exec events in single-threaded programs was a fairly
>> straightforward replication of the implementation in native GDB:
>>
>> 1) Enable exec events via ptrace options.
>>
>> 2) Add support for handling the exec events to the handle_extended_wait and
>> linux_wait_for_event_filtered.  Detect the exec event, then find and save
>> the pathname of the executable file being exec'd and set event status flags.
>>
>> 3) Implement an additional "stop reason", "exec", in the RSP stop reply
>> packet "T".
>>
>> Existing GDB code takes care of handling the exec event on the host side
>> without modification.
> 
> Hi Don,
> How does GDBserver handle the multi-arch case? say, 64-bit process call
> exec to a 32-bit program.  At least, the target description of that
> process in GDBserver should be updated.
> 
Hi Yao,
You make a good point, GDBserver doesn't handle that case.  I assume
that's what this is about:
---
warning: Selected architecture i386 is not compatible with reported
target architecture i386:x86-64
---
I'll investigate.
thanks
--Don

On 16/07/15 16:51, Don Breazeal wrote:
> You make a good point, GDBserver doesn't handle that case.  I assume
> that's what this is about:
> ---
> warning: Selected architecture i386 is not compatible with reported
> target architecture i386:x86-64
> ---
> I'll investigate.

This messages shows that GDB (rather than GDBserver) doesn't handle
that case.  AFAIK, GDBserver doesn't handle that case either.

I am working on patches create target description at the right time
in GDBserver, derived from this patch
https://sourceware.org/ml/gdb-patches/2015-07/msg00403.html
Current GDBserver creates target description too early, if we use
--wrapper option, GDBserver created target description according
to wrapper program, instead of the program we want to debug, which
is wrong.

On 7/16/2015 9:35 AM, Yao Qi wrote:
> On 16/07/15 16:51, Don Breazeal wrote:
>> You make a good point, GDBserver doesn't handle that case.  I assume
>> that's what this is about:
>> ---
>> warning: Selected architecture i386 is not compatible with reported
>> target architecture i386:x86-64
>> ---
>> I'll investigate.
> 
> This messages shows that GDB (rather than GDBserver) doesn't handle
> that case.  AFAIK, GDBserver doesn't handle that case either.
> 
> I am working on patches create target description at the right time
> in GDBserver, derived from this patch
> https://sourceware.org/ml/gdb-patches/2015-07/msg00403.html
> Current GDBserver creates target description too early, if we use
> --wrapper option, GDBserver created target description according
> to wrapper program, instead of the program we want to debug, which
> is wrong.
> 

There is a difference between the native and gdbserver behavior with
multi-arch exec.  Native seems to handle multi-arch exec events:
-------------------------------------------------------------------
Reading symbols from ./execler64...done.
(gdb) b main
Breakpoint 1 at 0x4006a4: file execler.c, line 19.
(gdb) r
Starting program: /home/dbreazea/junk/execler64

Breakpoint 1, main (argc=1, argv=0x7fffffffe848) at execler.c:19
19	  printf ("starting %s\n", argv[0]);
(gdb) info reg
rax            0x7ffff7dd9ea8	140737351884456
rbx            0x0	0
rcx            0x0	0
---etc---
(gdb) catch exec
Catchpoint 2 (exec)
(gdb) c
Continuing.
starting /home/dbreazea/junk/execler64
in execler
process 5588 is executing new program: /home/dbreazea/junk/execee32
warning: the debug information found in "/lib/ld-2.11.1.so" does not
match "/lib/ld-linux.so.2" (CRC mismatch).


Catchpoint 2 (exec'd /home/dbreazea/junk/execee32), 0xf7fe0850 in ?? ()
   from /lib/ld-linux.so.2
(gdb) info reg
eax            0x0	0
ecx            0x0	0
---etc---
---------------------------------------------------------------------

While the gdbserver case looks like this, unfortunately:

---------------------------------------------------------------------
Reading symbols from ./execler64...done.
(gdb) tar ext localhost:51111
Remote debugging using localhost:51111
Reading symbols from target:/lib64/ld-linux-x86-64.so.2...(no debugging
symbols found)...done.
0x00007ffff7dddaf0 in ?? () from target:/lib64/ld-linux-x86-64.so.2
(gdb) b main
Breakpoint 1 at 0x4006a4: file execler.c, line 19.
(gdb) c
Continuing.

Breakpoint 1, main (argc=1, argv=0x7fffffffe9d8) at execler.c:19
19	  printf ("starting %s\n", argv[0]);
(gdb) info reg
rax            0x7ffff7dd9ea8	140737351884456
rbx            0x0	0
rcx            0x0	0
---etc---
(gdb) catch exec
Catchpoint 2 (exec)
(gdb) c
Continuing.
Thread 5561.5561 is executing new program: /home/dbreazea/junk/execee32
warning: Selected architecture i386 is not compatible with reported
target architecture i386:x86-64
warning: the debug information found in "target:/lib/ld-2.11.1.so" does
not match "target:/lib/ld-linux.so.2" (CRC mismatch).

Remote 'g' packet reply is too long: 000000000000000000000000000000...
---several 'g' packet errors
(gdb) q
A debugging session is active.

	Inferior 1 [process 5561] will be killed.

Quit anyway? (y or n) y
warning: Selected architecture i386 is not compatible with reported
target architecture i386:x86-64
--------------------------------------------------------------------

At first glance it looks like in linux_low_filter_event, the execing
inferior needs to be marked as a 'new_inferior'
(proc->priv->new_inferior) in order to do the right thing and call
the_low_target.arch_setup ().  That may require some re-ordering of
things in linux-low.c:linux_low_filter_event, since handle_extended_wait
and the exec event handling happen after the arch setup.

Do you think the work you are doing will address this, or should I
continue looking at a fix for the problem above?  They seem like they
are related, but separate issues.

thanks
--Don

Don Breazeal <donb@codesourcery.com> writes:

> While the gdbserver case looks like this, unfortunately:
>
> ---------------------------------------------------------------------
> Reading symbols from ./execler64...done.
> (gdb) tar ext localhost:51111
> Remote debugging using localhost:51111
> Reading symbols from target:/lib64/ld-linux-x86-64.so.2...(no debugging
> symbols found)...done.
> 0x00007ffff7dddaf0 in ?? () from target:/lib64/ld-linux-x86-64.so.2
> (gdb) b main
> Breakpoint 1 at 0x4006a4: file execler.c, line 19.
> (gdb) c
> Continuing.
>
> Breakpoint 1, main (argc=1, argv=0x7fffffffe9d8) at execler.c:19
> 19	  printf ("starting %s\n", argv[0]);
> (gdb) info reg
> rax            0x7ffff7dd9ea8	140737351884456
> rbx            0x0	0
> rcx            0x0	0
> ---etc---
> (gdb) catch exec
> Catchpoint 2 (exec)
> (gdb) c
> Continuing.
> Thread 5561.5561 is executing new program: /home/dbreazea/junk/execee32
> warning: Selected architecture i386 is not compatible with reported
> target architecture i386:x86-64
> warning: the debug information found in "target:/lib/ld-2.11.1.so" does
> not match "target:/lib/ld-linux.so.2" (CRC mismatch).
>
> Remote 'g' packet reply is too long: 000000000000000000000000000000...
> ---several 'g' packet errors
> (gdb) q
> A debugging session is active.
>
> 	Inferior 1 [process 5561] will be killed.
>
> Quit anyway? (y or n) y
> warning: Selected architecture i386 is not compatible with reported
> target architecture i386:x86-64
> --------------------------------------------------------------------
>

I thought gdb.multi/multi-arch-exec.exp has already covered the case
above.  It fails in my clean GDB build.  If it doesn't cover, we need to
improve it or write a new one to cover the case.

> At first glance it looks like in linux_low_filter_event, the execing
> inferior needs to be marked as a 'new_inferior'
> (proc->priv->new_inferior) in order to do the right thing and call
> the_low_target.arch_setup ().  That may require some re-ordering of
> things in linux-low.c:linux_low_filter_event, since handle_extended_wait
> and the exec event handling happen after the arch setup.

Yes, this needs some re-ordering...

>
> Do you think the work you are doing will address this, or should I
> continue looking at a fix for the problem above?  They seem like they
> are related, but separate issues.

I think my patch series will address this, and I am testing them.  In
the mean time, I don't think this multi-arch issue blocks the review to
this patch series.  As you said, they are related, but separated issues.