diff mbox series

[5/6] gdb: add inferior-specific breakpoints and watchpoints

Message ID 0d3144a162d93c2b7d409234bd199e3c9fbbfce8.1669634536.git.aburgess@redhat.com
State New
Headers
Series Inferior specific breakpoints |

Commit Message

Andrew Burgess Nov. 28, 2022, 11:25 a.m. UTC 
  This commit extends the breakpoint mechanism to allow for inferior
specific breakpoints (and watchpoints).

As GDB gains better support for multiple connections, and so for
running multiple (possibly unrelated) inferiors, then it is not hard
to imagine that a user might wish to create breakpoints that apply to
any thread in a single inferior.  To achieve this currently, the user
would need to create a condition possibly making use of the $_inferior
convenience variable, which, though functional, isn't the most user
friendly.

This commit adds a new 'inferior' keyword that allows for the creation
of inferior specific breakpoints.

Like thread specific breakpoints, the inferior specific breakpoints
are automatically deleted once the inferior in question exits.
---
 gdb/NEWS                                      |   7 +
 gdb/breakpoint.c                              | 177 ++++++++++++++---
 gdb/breakpoint.h                              |  10 +-
 gdb/doc/gdb.texinfo                           |  74 ++++++-
 gdb/doc/python.texi                           |  27 ++-
 gdb/dummy-frame.c                             |   1 +
 gdb/elfread.c                                 |   5 +-
 gdb/guile/scm-breakpoint.c                    |   5 +
 gdb/inferior.h                                |  10 +
 gdb/infrun.c                                  |   2 +
 gdb/linespec.c                                |   4 +-
 gdb/python/py-breakpoint.c                    |  77 ++++++++
 gdb/testsuite/gdb.linespec/cpcompletion.exp   |   4 +-
 gdb/testsuite/gdb.linespec/explicit.exp       |   1 +
 .../gdb.multi/inferior-specific-bp-1.c        |  52 +++++
 .../gdb.multi/inferior-specific-bp-2.c        |  52 +++++
 .../gdb.multi/inferior-specific-bp.exp        | 183 ++++++++++++++++++
 gdb/testsuite/gdb.python/py-breakpoint.exp    |  43 ++++
 gdb/testsuite/lib/completion-support.exp      |   2 +-
 19 files changed, 688 insertions(+), 48 deletions(-)
 create mode 100644 gdb/testsuite/gdb.multi/inferior-specific-bp-1.c
 create mode 100644 gdb/testsuite/gdb.multi/inferior-specific-bp-2.c
 create mode 100644 gdb/testsuite/gdb.multi/inferior-specific-bp.exp

Comments

Eli Zaretskii Nov. 28, 2022, 1:18 p.m. UTC | #1 
> Cc: Andrew Burgess <aburgess@redhat.com>
> Date: Mon, 28 Nov 2022 11:25:37 +0000
> From: Andrew Burgess via Gdb-patches <gdb-patches@sourceware.org>
> 
> diff --git a/gdb/NEWS b/gdb/NEWS
> index 1f2233082ae..f3cb0f96f67 100644
> --- a/gdb/NEWS
> +++ b/gdb/NEWS
> @@ -110,6 +110,13 @@
>    instance of the keyword.  The 'thread' keyword already gave an error
>    when used multiple times with the watch command.
>  
> +* Breakpoints can now be inferior-specific.  This is similar to the
> +  existing thread-specific breakpoint support.  Breakpoint conditions
> +  can include the 'inferior' keyword followed by an inferior id (as
> +  displayed in the 'info inferiors' output).  It is invalid to use
> +  both the 'inferior' and 'thread' keywords when creating a
> +  breakpoint.
> +

This part is OK.

> +@node Inferior-Specific Breakpoints
> +@subsection Inferior-Specific Breakpoints
> +
> +When debugging multiple inferiors, you can choose whether to set
> +breakpoints for all inferiors, or for a particular inferior.
> +
> +@table @code
> +@cindex breakpoints and inferiors
> +@cindex inferior breakpoints

The last one should be "@cindex inferior-specific breakpoints", since
"inferior breakpoints" can be interpreted in a very confusing way.

> +Inferior-specific breakpoints are automatically deleted when
> +@value{GDBN} detects the corresponding inferior has exited.  For

"detects that the corresponding inferior has exited."  The "that" part is
missing.

> +A breakpoint can't be both thread-specific and inferior-specific
> +(@pxref{Inferior-Specific Breakpoints}) using both the @code{thread}
                                         ^^
Please add a semi-colon there.

> --- a/gdb/doc/python.texi
> +++ b/gdb/doc/python.texi
> @@ -3281,7 +3281,9 @@
>  A @code{gdb.Inferior} object has the following attributes:
>  
>  @defvar Inferior.num
> -ID of inferior, as assigned by GDB.
> +ID of inferior, as assigned by GDB.  You can use this to
                                  ^^^
"@value{GDBN}"

>  @anchor{python_breakpoint_thread}
>  @defvar Breakpoint.thread
> -If the breakpoint is thread-specific, this attribute holds the
> -thread's global id.  If the breakpoint is not thread-specific, this
> -attribute is @code{None}.  This attribute is writable.
> +If the breakpoint is thread-specific (@pxref{Thread-Specific
> +Breakpoints}), this attribute holds the thread's global id.  If the
> +breakpoint is not thread-specific, this attribute is @code{None}.
> +This attribute is writable.
> +
> +Only one of @code{Breakpoint.thread} or @code{Breakpoint.inferior} can
> +be set to a valid id at any time, that is, a breakpoint can be thread
> +specific, or inferior specific, but not both.
> +@end defvar
> +
> +@anchor{python_breakpoint_inferior}
> +@defvar Breakpoint.inferior
> +If the breakpoint is inferior-specific (@pxref{Inferior-Specific
> +Breakpoints}), this attribute holds the inferior's id.  If the
> +breakpoint is not inferior-specific, this attribute is @code{None}.
> +This attribute is writable.
> +
> +Only one of @code{Breakpoint.thread} or @code{Breakpoint.inferior} can
> +be set to a valid id at any time, that is, a breakpoint can be thread
> +specific, or inferior specific, but not both.

Please only keep the second paragraph of the two that start with "Only one".
It is redundant to have two identical paragraphs so close to one another.

The documentation part is OK with these nits fixed.

Thanks.
Terekhov, Mikhail via Gdb-patches Dec. 23, 2022, 10:05 a.m. UTC | #2 
On Monday, November 28, 2022 12:26 PM, Andrew Burgess wrote:
> This commit extends the breakpoint mechanism to allow for inferior
> specific breakpoints (and watchpoints).
> 
> As GDB gains better support for multiple connections, and so for
> running multiple (possibly unrelated) inferiors, then it is not hard

Nit: IMHO, removing "then" makes the sentence sound better.

> to imagine that a user might wish to create breakpoints that apply to
> any thread in a single inferior.  To achieve this currently, the user
> would need to create a condition possibly making use of the $_inferior
> convenience variable, which, though functional, isn't the most user
> friendly.

An important difference of an inferior-specific breakpoint wrt using
conditions that contain the 'thread' keyword or the '$_inferior' variable
could be that the breakpoints would not be inserted at all on other inferiors.
For inferiors that have a large number of threads, this could save a
substantial amount of overhead of stopping, evaluating the condition, and
resuming.  IMHO, it is worth considering this for inferior-specific breakpoints.

In a downstream debugger, we had included this feature:
https://github.com/intel/gdb/commit/7d87ac91308cd7a8984ba7b0e333a6689790972d
(Please see the modifications in 'create_breakpoint').

With this perspective, I also think that allowing the use of both 'thread'
and 'inferior' clauses makes sense, because they would have different advantages.

More comments are inlined below.

Thanks
-Baris
 
> @@ -3255,6 +3278,32 @@ Thread-specific breakpoint %d deleted - thread %s no longer in the
> thread list.\
>      }
>  }
> 
> +/* Called when inferior INF has exited.  Remove per-inferior breakpoints.  */
> +
> +static void
> +remove_inferior_breakpoints (struct inferior *inf)
> +{
> +  for (breakpoint *b : all_breakpoints_safe ())
> +    {
> +      if (b->inferior == inf->num && user_breakpoint_p (b))
> +	{
> +	  /* Tell the user the breakpoint has been deleted.  But only for
> +	     breakpoints that would not normally have been deleted at the
> +	     next stop anyway.  */
> +	  if (b->disposition != disp_del
> +	      && b->disposition != disp_del_at_next_stop)
> +	    gdb_printf (_("\
> +Inferior-specific breakpoint %d deleted - inferior %d has exited.\n"),
> +			b->number, inf->num);
> +
> +

It seems one of the blank lines is redundant.

> @@ -8430,6 +8500,11 @@ code_breakpoint::code_breakpoint (struct gdbarch *gdbarch_,
> 
>    thread = thread_;
>    task = task_;
> +  inferior = inferior_;
> +
> +  /* A breakpoint can be thread specific, or inferior specific, but not
> +     both.  This should be checked when the breakpoint condition is parsed.  */
> +  gdb_assert (!(thread != -1 && inferior != -1));

The previous assertion expressions are easier to read, I think:

  gdb_assert (thread == -1 || inferior == -1);

> @@ -8821,11 +8914,15 @@ find_condition_and_thread (const char *tok, CORE_ADDR pc,
>        else if (rest)
>  	{
>  	  rest->reset (savestring (tok, strlen (tok)));
> -	  return;
> +	  break;
>  	}
>        else
>  	error (_("Junk at end of arguments."));
>      }
> +
> +  if (*thread != -1 && *inferior != -1)
> +    error (_("Invalid use of both 'thread' and 'inferior' in "
> +	     "breakpoint condition"));

Nit: This error message sounds to me like "the conditions you used
both in the 'thread' and the 'inferior' clauses are incorrect."  But 
in fact the problem is, using both clauses at the same time is not
allowed.  Maybe this would be clearer:
"Using the 'thread' and 'inferior' conditions together is not allowed."

> @@ -10068,6 +10170,10 @@ watch_command_1 (const char *arg, int accessflag, int from_tty,
>  	  tok++;
>  	  toklen = end_tok - tok + 1;
> 
> +	  if (thread != -1 && inferior != -1)
> +	    error (_("Invalid use of both 'thread' and 'inferior' in "
> +		     "watchpoint condition"));

Same comment here.

> diff --git a/gdb/inferior.h b/gdb/inferior.h
> index 69525a2e053..3b25403d83c 100644
> --- a/gdb/inferior.h
> +++ b/gdb/inferior.h
> @@ -746,4 +746,14 @@ extern void print_selected_inferior (struct ui_out *uiout);
>  extern void switch_to_inferior_and_push_target
>    (inferior *new_inf, bool no_connection, inferior *org_inf);
> 
> +/* Return true if ID is a valid global inferior number.  */
> +
> +inline bool valid_global_inferior_id (int id)

Function name should be at column 0.

> @@ -280,11 +280,69 @@ bppy_set_thread (PyObject *self, PyObject *newvalue, void *closure)
>        return -1;
>      }
> 
> +  if (self_bp->bp->inferior != -1 && id != -1)
> +    {
> +      PyErr_SetString (PyExc_RuntimeError,
> +		       _("Cannot have both thread and inferior conditions "
> +			 "on a breakpoint"));

This error message is clearer than the one I commented above and I think
it would make sense to align the two.

> diff --git a/gdb/testsuite/gdb.multi/inferior-specific-bp.exp
> b/gdb/testsuite/gdb.multi/inferior-specific-bp.exp
> new file mode 100644
> index 00000000000..5d65f19b88c
> --- /dev/null
> +++ b/gdb/testsuite/gdb.multi/inferior-specific-bp.exp
> @@ -0,0 +1,183 @@
> +# Copyright 2022 Free Software Foundation, Inc.
> +
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 3 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
> +
> +# Test inferior-specific breakpoints.
> +
> +standard_testfile -1.c -2.c
> +
> +if { [use_gdb_stub] } {
> +    return
> +}
> +
> +set srcfile1 ${srcfile}
> +set binfile1 ${binfile}-1
> +set binfile2 ${binfile}-2
> +
> +if { [build_executable ${testfile}.exp ${binfile1} "${srcfile1}"] != 0 } {
> +    return -1
> +}
> +
> +if { [build_executable ${testfile}.exp ${binfile2} "${srcfile2}"] != 0 } {
> +    return -1
> +}
> +
> +# Start the first inferior.
> +clean_restart ${binfile1}
> +if {![runto_main]} {

In the if-statements above, there are spaces around the condition.

> +    return
> +}
> +
> +# Add a second inferior, and start this one too.
> +gdb_test "add-inferior" "Added inferior 2.*" "add empty inferior 2"
> +gdb_test "inferior 2" "Switching to inferior 2.*" "switch to inferior 2"
> +gdb_load $binfile2
> +if {![runto_main]} {
> +    return
> +}
> +
> +# Try to create a breakpoint using both the 'inferior' and 'thread' keywords,
> +# this should fail.  Try with the keywords in both orders just in case the
> +# parser has a bug.
> +gdb_test "break foo thread 1.1 inferior 1" \
> +    "Invalid use of both 'thread' and 'inferior' in breakpoint condition"
> +gdb_test "break foo inferior 1 thread 1.1" \
> +    "Invalid use of both 'thread' and 'inferior' in breakpoint condition"
> +
> +# While w're here, check that we can't create a watchpoint with both thread

Typo: w're

> +# and inferior keywords.  As above, test with keywords in both orders.
> +foreach type {watch rwatch awatch} {
> +    gdb_test "$type global_var thread 1.1 inferior 1" \
> +	"Invalid use of both 'thread' and 'inferior' in watchpoint condition"
> +    gdb_test "$type global_var inferior 1 thread 1.1" \
> +	"Invalid use of both 'thread' and 'inferior' in watchpoint condition"
> +}
> +
> +# Clear out any other breakpoints.
> +gdb_test "with confirm off -- delete breakpoints"

Why not use "delete_breakpoints"?

> +# Create an inferior specific breakpoint.

Nit: This should be spelled "inferior-specific", as far as I know.

> +gdb_test "break foo inferior 1" \
> +    "Breakpoint $decimal at $hex: foo\\. \\(3 locations\\)"

Why not use 'gdb_breakpoint'?

> +set saw_header false
> +set location_count 0
> +set saw_inf_cond false
> +gdb_test_multiple "info breakpoints" "" {
> +    -re "^info breakpoints\r\n" {
> +	exp_continue
> +    }
> +
> +    -re "^Num\\s+\[^\r\n\]+\r\n" {

Removing "^" at the beginning could help eliminate the first regexp
above to obtain shorter code.

> +	exp_continue
> +    }
> +
> +    -re "^$decimal\\s+breakpoint\\s+keep\\s+y\\s+<MULTIPLE>\\s*\r\n" {
> +	set saw_header true
> +	exp_continue
> +    }
> +
> +    -re "^\\s+stop only in inferior 1\r\n" {
> +	set saw_inf_cond true
> +	exp_continue
> +    }
> +
> +    -re "^$decimal\\.\[123\]\\s+y\\s+ $hex in foo at \[^\r\n\]+ inf \[12\] inferior 1\r\n"
> {
> +	incr location_count
> +	exp_continue
> +    }
> +
> +    -re "^$gdb_prompt $" {
> +	with_test_prefix $gdb_test_name {
> +	    gdb_assert { $saw_header }
> +	    gdb_assert { $location_count == 3 }
> +	    gdb_assert { $saw_inf_cond }
> +	}
> +    }
> +}
> +
> +# Create a multi-inferior breakpoint to stop at.
> +gdb_test "break stop_breakpt"
> +
> +# Now resume inferior 2, this should reach 'stop_breakpt'.
> +gdb_test "continue" "hit Breakpoint $decimal\.$decimal, stop_breakpt \\(\\) .*" \
> +    "continue in inferior 2"
> +
> +# Switch to inferior 1, and try there.
> +gdb_test "inferior 1" ".*" \
> +    "select inferior 1 to check the inferior-specific b/p works"
> +gdb_test "continue " "hit Breakpoint $decimal\.$decimal, foo \\(\\) .*" \
> +    "first continue in inferior 1"

Maybe we can check here explicitly, for extra confidence, that the hit
came actually from Thread 1.*.

> +
> +# Now back to inferior 2, let the inferior exit, the inferior-specific
> +# breakpoint should not be deleted.
> +gdb_test "inferior 2" ".*" \
> +    "switch back to allow inferior 2 to exit"
> +gdb_test "continue" "Inferior 2 \[^\r\n\]+ exited normally.*" \
> +    "allow inferior 2 to exit"
> +
> +gdb_test "inferior 1" ".*" \
> +    "select inferior 1 to check inferior-specific b/p still works"
> +gdb_test "continue " "hit Breakpoint $decimal\.$decimal, foo \\(\\) .*" \
> +    "second continue in inferior 1"
> +gdb_test "continue " "hit Breakpoint $decimal\.$decimal, stop_breakpt \\(\\) .*" \
> +    "third continue in inferior 1"
> +
> +# Now allow inferior 1 to exit, the inferior specific breakpoint should be
> +# deleted.
> +gdb_test "continue" \
> +    [multi_line \
> +	 "\\\[Inferior 1 \[^\r\n\]+ exited normally\\\]" \
> +	 "Inferior-specific breakpoint $decimal deleted - inferior 1 has exited\\."]
> +
> +set saw_header false
> +set location_count 0
> +set saw_inf_cond false
> +gdb_test_multiple "info breakpoints" "info breakpoint after inferior 1 exited" {
> +    -re "^info breakpoints\r\n" {
> +	exp_continue
> +    }
> +
> +    -re "^Num\\s+\[^\r\n\]+\r\n" {
> +	exp_continue
> +    }
> +
> +    -re "^$decimal\\s+breakpoint\\s+keep\\s+y\\s+<MULTIPLE>\\s*\r\n" {
> +	set saw_header true
> +	exp_continue
> +    }
> +
> +    -re "^\\s+stop only in inferior 1\r\n" {
> +	# This should not happen, this breakpoint should have been deleted.
> +	set saw_inf_cond true
> +	exp_continue
> +    }
> +
> +    -re "^\\s+breakpoint already hit 2 times\r\n" {
> +	exp_continue
> +    }
> +
> +    -re "^$decimal\\.\[12\]\\s+y\\s+ $hex in stop_breakpt at \[^\r\n\]+ inf \[12\]\r\n" {
> +	incr location_count
> +	exp_continue
> +    }
> +
> +    -re "^$gdb_prompt $" {
> +	with_test_prefix $gdb_test_name {
> +	    gdb_assert { $saw_header }
> +	    gdb_assert { $location_count == 2 }
> +	    gdb_assert { !$saw_inf_cond }
> +	}
> +    }
> +}
> diff --git a/gdb/testsuite/gdb.python/py-breakpoint.exp b/gdb/testsuite/gdb.python/py-
> breakpoint.exp
> index c3215b13d5c..8906dea6655 100644
> --- a/gdb/testsuite/gdb.python/py-breakpoint.exp
> +++ b/gdb/testsuite/gdb.python/py-breakpoint.exp
> @@ -112,6 +112,8 @@ proc_with_prefix test_bkpt_basic { } {
>  	"Get Breakpoint List" 0
>      gdb_test "python print (blist\[1\].thread)" \
>  	"None" "Check breakpoint thread"
> +    gdb_test "python print (blist\[1\].inferior)" \
> +	"None" "Check breakpoint inferior"
>      gdb_test "python print (blist\[1\].type == gdb.BP_BREAKPOINT)" \
>  	"True" "Check breakpoint type"
>      gdb_test "python print (blist\[0\].number)" \
> @@ -214,6 +216,46 @@ proc_with_prefix test_bkpt_cond_and_cmds { } {
>  	"check number of lines in commands"
>  }
> 
> +# Test breakpoint thread and inferior attributes.
> +proc_with_prefix test_bkpt_thread_and_inferior { } {
> +    global srcfile testfile hex decimal
> +
> +    # Start with a fresh gdb.
> +    clean_restart ${testfile}
> +
> +    if ![runto_main] then {

I thought you had removed the uses of 'then'. :)

> +	return 0

As I wrote previously, the testsuite does not look consistent
about what to return here.  But "-1" makes more sense to me, because
not being able to run to main sounds like a major problem.

> +    }
> +
> +    with_test_prefix "thread" {
> +	delete_breakpoints
> +	gdb_test "break multiply thread 1"
> +	gdb_test "python bp = gdb.breakpoints ()\[0\]"
> +	gdb_test "python print(bp.thread)" "1"
> +	gdb_test "python print(bp.inferior)" "None"
> +	gdb_test "python bp.inferior = 1" \
> +	    "RuntimeError: Cannot have both thread and inferior conditions on a breakpoint.*"
> +	gdb_test_no_output "python bp.thread = None"
> +	gdb_test_no_output "python bp.inferior = 1" \
> +	    "set the inferior now the thread has been cleared"
> +	gdb_test "info breakpoints" "stop only in inferior 1\r\n.*"
> +    }
> +
> +    with_test_prefix "inferior" {
> +	delete_breakpoints
> +	gdb_test "break multiply inferior 1"
> +	gdb_test "python bp = gdb.breakpoints ()\[0\]"
> +	gdb_test "python print(bp.thread)" "None"
> +	gdb_test "python print(bp.inferior)" "1"
> +	gdb_test "python bp.thread = 1" \
> +	    "RuntimeError: Cannot have both thread and inferior conditions on a breakpoint.*"
> +	gdb_test_no_output "python bp.inferior = None"
> +	gdb_test_no_output "python bp.thread = 1" \
> +	    "set the thread now the inferior has been cleared"
> +	gdb_test "info breakpoints" "stop only in thread 1\r\n.*"
> +    }
> +}
> +
>  proc_with_prefix test_bkpt_invisible { } {
>      global srcfile testfile hex decimal
> 
> @@ -849,6 +891,7 @@ proc_with_prefix test_bkpt_auto_disable { } {
>  test_bkpt_basic
>  test_bkpt_deletion
>  test_bkpt_cond_and_cmds
> +test_bkpt_thread_and_inferior
>  test_bkpt_invisible
>  test_hardware_breakpoints
>  test_catchpoints
> diff --git a/gdb/testsuite/lib/completion-support.exp b/gdb/testsuite/lib/completion-
> support.exp
> index 0c03d0f035e..135649ed4d9 100644
> --- a/gdb/testsuite/lib/completion-support.exp
> +++ b/gdb/testsuite/lib/completion-support.exp
> @@ -27,7 +27,7 @@ namespace eval completion {
>      # List of all quote chars, including no-quote at all.
>      variable maybe_quoted_list {"" "'" "\""}
> 
> -    variable keyword_list {"-force-condition" "if" "task" "thread"}
> +    variable keyword_list {"-force-condition" "if" "inferior" "task" "thread"}
> 
>      variable explicit_opts_list \
>  	{"-function" "-label" "-line" "-qualified" "-source"}
> --
> 2.25.4

Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de <http://www.intel.de>
Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva  
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928
Andrew Burgess Jan. 19, 2023, 7:13 p.m. UTC | #3 
"Aktemur, Tankut Baris" <tankut.baris.aktemur@intel.com> writes:

> On Monday, November 28, 2022 12:26 PM, Andrew Burgess wrote:
>> This commit extends the breakpoint mechanism to allow for inferior
>> specific breakpoints (and watchpoints).
>> 
>> As GDB gains better support for multiple connections, and so for
>> running multiple (possibly unrelated) inferiors, then it is not hard
>
> Nit: IMHO, removing "then" makes the sentence sound better.
>
>> to imagine that a user might wish to create breakpoints that apply to
>> any thread in a single inferior.  To achieve this currently, the user
>> would need to create a condition possibly making use of the $_inferior
>> convenience variable, which, though functional, isn't the most user
>> friendly.
>
> An important difference of an inferior-specific breakpoint wrt using
> conditions that contain the 'thread' keyword or the '$_inferior' variable
> could be that the breakpoints would not be inserted at all on other inferiors.
> For inferiors that have a large number of threads, this could save a
> substantial amount of overhead of stopping, evaluating the condition, and
> resuming.  IMHO, it is worth considering this for inferior-specific breakpoints.
>
> In a downstream debugger, we had included this feature:
> https://github.com/intel/gdb/commit/7d87ac91308cd7a8984ba7b0e333a6689790972d
> (Please see the modifications in 'create_breakpoint').

Neat.  I wasn't aware Intel had already worked on this feature.

I had also thought about not inserting breakpoints into non-matching
inferiors.  In the end I decided to leave that for a follow up patch,
but it's nice to see that Intel have been doing this for a few years now.

>
> With this perspective, I also think that allowing the use of both 'thread'
> and 'inferior' clauses makes sense, because they would have different advantages.

Except, isn't the thread-id passed to a 'thread' condition a global
thread-id?  i.e. "break foo thread 1" isn't thread 1 in every inferior,
it's GDB's global thread 1, which is one thread in one inferior.

So we could (if we implemented it) already limit into which inferiors a
thread specific breakpoint is inserted by just figuring out which
inferior that thread is in.

I think it makes sense, at least initially, to prevent use of 'thread'
and 'inferior' together.  If we decide to relax this restriction later,
then that's no problem.  It's much harder to add more restrictions
later.

>
> More comments are inlined below.
>
> Thanks
> -Baris
>  
>> @@ -3255,6 +3278,32 @@ Thread-specific breakpoint %d deleted - thread %s no longer in the
>> thread list.\
>>      }
>>  }
>> 
>> +/* Called when inferior INF has exited.  Remove per-inferior breakpoints.  */
>> +
>> +static void
>> +remove_inferior_breakpoints (struct inferior *inf)
>> +{
>> +  for (breakpoint *b : all_breakpoints_safe ())
>> +    {
>> +      if (b->inferior == inf->num && user_breakpoint_p (b))
>> +	{
>> +	  /* Tell the user the breakpoint has been deleted.  But only for
>> +	     breakpoints that would not normally have been deleted at the
>> +	     next stop anyway.  */
>> +	  if (b->disposition != disp_del
>> +	      && b->disposition != disp_del_at_next_stop)
>> +	    gdb_printf (_("\
>> +Inferior-specific breakpoint %d deleted - inferior %d has exited.\n"),
>> +			b->number, inf->num);
>> +
>> +
>
> It seems one of the blank lines is redundant.
>
>> @@ -8430,6 +8500,11 @@ code_breakpoint::code_breakpoint (struct gdbarch *gdbarch_,
>> 
>>    thread = thread_;
>>    task = task_;
>> +  inferior = inferior_;
>> +
>> +  /* A breakpoint can be thread specific, or inferior specific, but not
>> +     both.  This should be checked when the breakpoint condition is parsed.  */
>> +  gdb_assert (!(thread != -1 && inferior != -1));
>
> The previous assertion expressions are easier to read, I think:
>
>   gdb_assert (thread == -1 || inferior == -1);
>
>> @@ -8821,11 +8914,15 @@ find_condition_and_thread (const char *tok, CORE_ADDR pc,
>>        else if (rest)
>>  	{
>>  	  rest->reset (savestring (tok, strlen (tok)));
>> -	  return;
>> +	  break;
>>  	}
>>        else
>>  	error (_("Junk at end of arguments."));
>>      }
>> +
>> +  if (*thread != -1 && *inferior != -1)
>> +    error (_("Invalid use of both 'thread' and 'inferior' in "
>> +	     "breakpoint condition"));
>
> Nit: This error message sounds to me like "the conditions you used
> both in the 'thread' and the 'inferior' clauses are incorrect."  But 
> in fact the problem is, using both clauses at the same time is not
> allowed.  Maybe this would be clearer:
> "Using the 'thread' and 'inferior' conditions together is not allowed."
>
>> @@ -10068,6 +10170,10 @@ watch_command_1 (const char *arg, int accessflag, int from_tty,
>>  	  tok++;
>>  	  toklen = end_tok - tok + 1;
>> 
>> +	  if (thread != -1 && inferior != -1)
>> +	    error (_("Invalid use of both 'thread' and 'inferior' in "
>> +		     "watchpoint condition"));
>
> Same comment here.
>
>> diff --git a/gdb/inferior.h b/gdb/inferior.h
>> index 69525a2e053..3b25403d83c 100644
>> --- a/gdb/inferior.h
>> +++ b/gdb/inferior.h
>> @@ -746,4 +746,14 @@ extern void print_selected_inferior (struct ui_out *uiout);
>>  extern void switch_to_inferior_and_push_target
>>    (inferior *new_inf, bool no_connection, inferior *org_inf);
>> 
>> +/* Return true if ID is a valid global inferior number.  */
>> +
>> +inline bool valid_global_inferior_id (int id)
>
> Function name should be at column 0.
>
>> @@ -280,11 +280,69 @@ bppy_set_thread (PyObject *self, PyObject *newvalue, void *closure)
>>        return -1;
>>      }
>> 
>> +  if (self_bp->bp->inferior != -1 && id != -1)
>> +    {
>> +      PyErr_SetString (PyExc_RuntimeError,
>> +		       _("Cannot have both thread and inferior conditions "
>> +			 "on a breakpoint"));
>
> This error message is clearer than the one I commented above and I think
> it would make sense to align the two.
>
>> diff --git a/gdb/testsuite/gdb.multi/inferior-specific-bp.exp
>> b/gdb/testsuite/gdb.multi/inferior-specific-bp.exp
>> new file mode 100644
>> index 00000000000..5d65f19b88c
>> --- /dev/null
>> +++ b/gdb/testsuite/gdb.multi/inferior-specific-bp.exp
>> @@ -0,0 +1,183 @@
>> +# Copyright 2022 Free Software Foundation, Inc.
>> +
>> +# This program is free software; you can redistribute it and/or modify
>> +# it under the terms of the GNU General Public License as published by
>> +# the Free Software Foundation; either version 3 of the License, or
>> +# (at your option) any later version.
>> +#
>> +# This program is distributed in the hope that it will be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
>> +
>> +# Test inferior-specific breakpoints.
>> +
>> +standard_testfile -1.c -2.c
>> +
>> +if { [use_gdb_stub] } {
>> +    return
>> +}
>> +
>> +set srcfile1 ${srcfile}
>> +set binfile1 ${binfile}-1
>> +set binfile2 ${binfile}-2
>> +
>> +if { [build_executable ${testfile}.exp ${binfile1} "${srcfile1}"] != 0 } {
>> +    return -1
>> +}
>> +
>> +if { [build_executable ${testfile}.exp ${binfile2} "${srcfile2}"] != 0 } {
>> +    return -1
>> +}
>> +
>> +# Start the first inferior.
>> +clean_restart ${binfile1}
>> +if {![runto_main]} {
>
> In the if-statements above, there are spaces around the condition.
>
>> +    return
>> +}
>> +
>> +# Add a second inferior, and start this one too.
>> +gdb_test "add-inferior" "Added inferior 2.*" "add empty inferior 2"
>> +gdb_test "inferior 2" "Switching to inferior 2.*" "switch to inferior 2"
>> +gdb_load $binfile2
>> +if {![runto_main]} {
>> +    return
>> +}
>> +
>> +# Try to create a breakpoint using both the 'inferior' and 'thread' keywords,
>> +# this should fail.  Try with the keywords in both orders just in case the
>> +# parser has a bug.
>> +gdb_test "break foo thread 1.1 inferior 1" \
>> +    "Invalid use of both 'thread' and 'inferior' in breakpoint condition"
>> +gdb_test "break foo inferior 1 thread 1.1" \
>> +    "Invalid use of both 'thread' and 'inferior' in breakpoint condition"
>> +
>> +# While w're here, check that we can't create a watchpoint with both thread
>
> Typo: w're
>
>> +# and inferior keywords.  As above, test with keywords in both orders.
>> +foreach type {watch rwatch awatch} {
>> +    gdb_test "$type global_var thread 1.1 inferior 1" \
>> +	"Invalid use of both 'thread' and 'inferior' in watchpoint condition"
>> +    gdb_test "$type global_var inferior 1 thread 1.1" \
>> +	"Invalid use of both 'thread' and 'inferior' in watchpoint condition"
>> +}
>> +
>> +# Clear out any other breakpoints.
>> +gdb_test "with confirm off -- delete breakpoints"
>
> Why not use "delete_breakpoints"?
>
>> +# Create an inferior specific breakpoint.
>
> Nit: This should be spelled "inferior-specific", as far as I know.
>
>> +gdb_test "break foo inferior 1" \
>> +    "Breakpoint $decimal at $hex: foo\\. \\(3 locations\\)"
>
> Why not use 'gdb_breakpoint'?
>
>> +set saw_header false
>> +set location_count 0
>> +set saw_inf_cond false
>> +gdb_test_multiple "info breakpoints" "" {
>> +    -re "^info breakpoints\r\n" {
>> +	exp_continue
>> +    }
>> +
>> +    -re "^Num\\s+\[^\r\n\]+\r\n" {
>
> Removing "^" at the beginning could help eliminate the first regexp
> above to obtain shorter code.
>
>> +	exp_continue
>> +    }
>> +
>> +    -re "^$decimal\\s+breakpoint\\s+keep\\s+y\\s+<MULTIPLE>\\s*\r\n" {
>> +	set saw_header true
>> +	exp_continue
>> +    }
>> +
>> +    -re "^\\s+stop only in inferior 1\r\n" {
>> +	set saw_inf_cond true
>> +	exp_continue
>> +    }
>> +
>> +    -re "^$decimal\\.\[123\]\\s+y\\s+ $hex in foo at \[^\r\n\]+ inf \[12\] inferior 1\r\n"
>> {
>> +	incr location_count
>> +	exp_continue
>> +    }
>> +
>> +    -re "^$gdb_prompt $" {
>> +	with_test_prefix $gdb_test_name {
>> +	    gdb_assert { $saw_header }
>> +	    gdb_assert { $location_count == 3 }
>> +	    gdb_assert { $saw_inf_cond }
>> +	}
>> +    }
>> +}
>> +
>> +# Create a multi-inferior breakpoint to stop at.
>> +gdb_test "break stop_breakpt"
>> +
>> +# Now resume inferior 2, this should reach 'stop_breakpt'.
>> +gdb_test "continue" "hit Breakpoint $decimal\.$decimal, stop_breakpt \\(\\) .*" \
>> +    "continue in inferior 2"
>> +
>> +# Switch to inferior 1, and try there.
>> +gdb_test "inferior 1" ".*" \
>> +    "select inferior 1 to check the inferior-specific b/p works"
>> +gdb_test "continue " "hit Breakpoint $decimal\.$decimal, foo \\(\\) .*" \
>> +    "first continue in inferior 1"
>
> Maybe we can check here explicitly, for extra confidence, that the hit
> came actually from Thread 1.*.
>
>> +
>> +# Now back to inferior 2, let the inferior exit, the inferior-specific
>> +# breakpoint should not be deleted.
>> +gdb_test "inferior 2" ".*" \
>> +    "switch back to allow inferior 2 to exit"
>> +gdb_test "continue" "Inferior 2 \[^\r\n\]+ exited normally.*" \
>> +    "allow inferior 2 to exit"
>> +
>> +gdb_test "inferior 1" ".*" \
>> +    "select inferior 1 to check inferior-specific b/p still works"
>> +gdb_test "continue " "hit Breakpoint $decimal\.$decimal, foo \\(\\) .*" \
>> +    "second continue in inferior 1"
>> +gdb_test "continue " "hit Breakpoint $decimal\.$decimal, stop_breakpt \\(\\) .*" \
>> +    "third continue in inferior 1"
>> +
>> +# Now allow inferior 1 to exit, the inferior specific breakpoint should be
>> +# deleted.
>> +gdb_test "continue" \
>> +    [multi_line \
>> +	 "\\\[Inferior 1 \[^\r\n\]+ exited normally\\\]" \
>> +	 "Inferior-specific breakpoint $decimal deleted - inferior 1 has exited\\."]
>> +
>> +set saw_header false
>> +set location_count 0
>> +set saw_inf_cond false
>> +gdb_test_multiple "info breakpoints" "info breakpoint after inferior 1 exited" {
>> +    -re "^info breakpoints\r\n" {
>> +	exp_continue
>> +    }
>> +
>> +    -re "^Num\\s+\[^\r\n\]+\r\n" {
>> +	exp_continue
>> +    }
>> +
>> +    -re "^$decimal\\s+breakpoint\\s+keep\\s+y\\s+<MULTIPLE>\\s*\r\n" {
>> +	set saw_header true
>> +	exp_continue
>> +    }
>> +
>> +    -re "^\\s+stop only in inferior 1\r\n" {
>> +	# This should not happen, this breakpoint should have been deleted.
>> +	set saw_inf_cond true
>> +	exp_continue
>> +    }
>> +
>> +    -re "^\\s+breakpoint already hit 2 times\r\n" {
>> +	exp_continue
>> +    }
>> +
>> +    -re "^$decimal\\.\[12\]\\s+y\\s+ $hex in stop_breakpt at \[^\r\n\]+ inf \[12\]\r\n" {
>> +	incr location_count
>> +	exp_continue
>> +    }
>> +
>> +    -re "^$gdb_prompt $" {
>> +	with_test_prefix $gdb_test_name {
>> +	    gdb_assert { $saw_header }
>> +	    gdb_assert { $location_count == 2 }
>> +	    gdb_assert { !$saw_inf_cond }
>> +	}
>> +    }
>> +}
>> diff --git a/gdb/testsuite/gdb.python/py-breakpoint.exp b/gdb/testsuite/gdb.python/py-
>> breakpoint.exp
>> index c3215b13d5c..8906dea6655 100644
>> --- a/gdb/testsuite/gdb.python/py-breakpoint.exp
>> +++ b/gdb/testsuite/gdb.python/py-breakpoint.exp
>> @@ -112,6 +112,8 @@ proc_with_prefix test_bkpt_basic { } {
>>  	"Get Breakpoint List" 0
>>      gdb_test "python print (blist\[1\].thread)" \
>>  	"None" "Check breakpoint thread"
>> +    gdb_test "python print (blist\[1\].inferior)" \
>> +	"None" "Check breakpoint inferior"
>>      gdb_test "python print (blist\[1\].type == gdb.BP_BREAKPOINT)" \
>>  	"True" "Check breakpoint type"
>>      gdb_test "python print (blist\[0\].number)" \
>> @@ -214,6 +216,46 @@ proc_with_prefix test_bkpt_cond_and_cmds { } {
>>  	"check number of lines in commands"
>>  }
>> 
>> +# Test breakpoint thread and inferior attributes.
>> +proc_with_prefix test_bkpt_thread_and_inferior { } {
>> +    global srcfile testfile hex decimal
>> +
>> +    # Start with a fresh gdb.
>> +    clean_restart ${testfile}
>> +
>> +    if ![runto_main] then {
>
> I thought you had removed the uses of 'then'. :)
>
>> +	return 0
>
> As I wrote previously, the testsuite does not look consistent
> about what to return here.  But "-1" makes more sense to me, because
> not being able to run to main sounds like a major problem.

The difference in the other patch where you pointed this out is that the
return as at the top level of the script (I fixed that other case as per
your suggestion).

In this case the return is from a function.  We don't actually check the
return value, but I'd rather leave this return consistent with all the
other similar return statements, in all the other functions, in this
test script.

I've made all the other updates you suggested, these will be included in
a V2 series shortly.

Thanks,
Andrew

>
>> +    }
>> +
>> +    with_test_prefix "thread" {
>> +	delete_breakpoints
>> +	gdb_test "break multiply thread 1"
>> +	gdb_test "python bp = gdb.breakpoints ()\[0\]"
>> +	gdb_test "python print(bp.thread)" "1"
>> +	gdb_test "python print(bp.inferior)" "None"
>> +	gdb_test "python bp.inferior = 1" \
>> +	    "RuntimeError: Cannot have both thread and inferior conditions on a breakpoint.*"
>> +	gdb_test_no_output "python bp.thread = None"
>> +	gdb_test_no_output "python bp.inferior = 1" \
>> +	    "set the inferior now the thread has been cleared"
>> +	gdb_test "info breakpoints" "stop only in inferior 1\r\n.*"
>> +    }
>> +
>> +    with_test_prefix "inferior" {
>> +	delete_breakpoints
>> +	gdb_test "break multiply inferior 1"
>> +	gdb_test "python bp = gdb.breakpoints ()\[0\]"
>> +	gdb_test "python print(bp.thread)" "None"
>> +	gdb_test "python print(bp.inferior)" "1"
>> +	gdb_test "python bp.thread = 1" \
>> +	    "RuntimeError: Cannot have both thread and inferior conditions on a breakpoint.*"
>> +	gdb_test_no_output "python bp.inferior = None"
>> +	gdb_test_no_output "python bp.thread = 1" \
>> +	    "set the thread now the inferior has been cleared"
>> +	gdb_test "info breakpoints" "stop only in thread 1\r\n.*"
>> +    }
>> +}
>> +
>>  proc_with_prefix test_bkpt_invisible { } {
>>      global srcfile testfile hex decimal
>> 
>> @@ -849,6 +891,7 @@ proc_with_prefix test_bkpt_auto_disable { } {
>>  test_bkpt_basic
>>  test_bkpt_deletion
>>  test_bkpt_cond_and_cmds
>> +test_bkpt_thread_and_inferior
>>  test_bkpt_invisible
>>  test_hardware_breakpoints
>>  test_catchpoints
>> diff --git a/gdb/testsuite/lib/completion-support.exp b/gdb/testsuite/lib/completion-
>> support.exp
>> index 0c03d0f035e..135649ed4d9 100644
>> --- a/gdb/testsuite/lib/completion-support.exp
>> +++ b/gdb/testsuite/lib/completion-support.exp
>> @@ -27,7 +27,7 @@ namespace eval completion {
>>      # List of all quote chars, including no-quote at all.
>>      variable maybe_quoted_list {"" "'" "\""}
>> 
>> -    variable keyword_list {"-force-condition" "if" "task" "thread"}
>> +    variable keyword_list {"-force-condition" "if" "inferior" "task" "thread"}
>> 
>>      variable explicit_opts_list \
>>  	{"-function" "-label" "-line" "-qualified" "-source"}
>> --
>> 2.25.4
>
> Intel Deutschland GmbH
> Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
> Tel: +49 89 99 8853-0, www.intel.de <http://www.intel.de>
> Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva  
> Chairperson of the Supervisory Board: Nicole Lau
> Registered Office: Munich
> Commercial Register: Amtsgericht Muenchen HRB 186928
Terekhov, Mikhail via Gdb-patches Jan. 20, 2023, 1:12 p.m. UTC | #4 
On Thursday, January 19, 2023 8:14 PM, Andrew Burgess wrote:
> "Aktemur, Tankut Baris" <tankut.baris.aktemur@intel.com> writes:
> 
> > On Monday, November 28, 2022 12:26 PM, Andrew Burgess wrote:
> >> This commit extends the breakpoint mechanism to allow for inferior
> >> specific breakpoints (and watchpoints).
> >>
> >> As GDB gains better support for multiple connections, and so for
> >> running multiple (possibly unrelated) inferiors, then it is not hard
> >
> > Nit: IMHO, removing "then" makes the sentence sound better.
> >
> >> to imagine that a user might wish to create breakpoints that apply to
> >> any thread in a single inferior.  To achieve this currently, the user
> >> would need to create a condition possibly making use of the $_inferior
> >> convenience variable, which, though functional, isn't the most user
> >> friendly.
> >
> > An important difference of an inferior-specific breakpoint wrt using
> > conditions that contain the 'thread' keyword or the '$_inferior' variable
> > could be that the breakpoints would not be inserted at all on other inferiors.
> > For inferiors that have a large number of threads, this could save a
> > substantial amount of overhead of stopping, evaluating the condition, and
> > resuming.  IMHO, it is worth considering this for inferior-specific breakpoints.
> >
> > In a downstream debugger, we had included this feature:
> > https://github.com/intel/gdb/commit/7d87ac91308cd7a8984ba7b0e333a6689790972d
> > (Please see the modifications in 'create_breakpoint').
> 
> Neat.  I wasn't aware Intel had already worked on this feature.
> 
> I had also thought about not inserting breakpoints into non-matching
> inferiors.  In the end I decided to leave that for a follow up patch,
> but it's nice to see that Intel have been doing this for a few years now.

We should&could have submitted the patch to upstream earlier, but this was
delayed due to several factors.  Sorry about that.  This could have saved some
efforts.

Avoiding the insertion of the bp in other inferiors can certainly be done
in a follow-up, IMHO.

> >
> > With this perspective, I also think that allowing the use of both 'thread'
> > and 'inferior' clauses makes sense, because they would have different advantages.
> 
> Except, isn't the thread-id passed to a 'thread' condition a global
> thread-id?  i.e. "break foo thread 1" isn't thread 1 in every inferior,
> it's GDB's global thread 1, which is one thread in one inferior.
>
> So we could (if we implemented it) already limit into which inferiors a
> thread specific breakpoint is inserted by just figuring out which
> inferior that thread is in.

Yes, this would make sense.
 
> I think it makes sense, at least initially, to prevent use of 'thread'
> and 'inferior' together.  If we decide to relax this restriction later,
> then that's no problem.  It's much harder to add more restrictions
> later.

Ok, that's right.

...
> >> +	return 0
> >
> > As I wrote previously, the testsuite does not look consistent
> > about what to return here.  But "-1" makes more sense to me, because
> > not being able to run to main sounds like a major problem.
> 
> The difference in the other patch where you pointed this out is that the
> return as at the top level of the script (I fixed that other case as per
> your suggestion).
> 
> In this case the return is from a function.  We don't actually check the
> return value, but I'd rather leave this return consistent with all the
> other similar return statements, in all the other functions, in this
> test script.

I missed that this was returning from a function.  I see your point and it
makes sense.

> I've made all the other updates you suggested, these will be included in
> a V2 series shortly.

I'll go over the series once more as soon as I can.

Thanks,
-Baris


Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de <http://www.intel.de>
Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva  
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928
diff mbox series

Patch

diff --git a/gdb/NEWS b/gdb/NEWS
index 1f2233082ae..f3cb0f96f67 100644
--- a/gdb/NEWS
+++ b/gdb/NEWS
@@ -110,6 +110,13 @@ 
   instance of the keyword.  The 'thread' keyword already gave an error
   when used multiple times with the watch command.
 
+* Breakpoints can now be inferior-specific.  This is similar to the
+  existing thread-specific breakpoint support.  Breakpoint conditions
+  can include the 'inferior' keyword followed by an inferior id (as
+  displayed in the 'info inferiors' output).  It is invalid to use
+  both the 'inferior' and 'thread' keywords when creating a
+  breakpoint.
+
 * New commands
 
 maintenance set ignore-prologue-end-flag on|off
diff --git a/gdb/breakpoint.c b/gdb/breakpoint.c
index bf99297a250..307f4e7720f 100644
--- a/gdb/breakpoint.c
+++ b/gdb/breakpoint.c
@@ -96,7 +96,7 @@  static void create_breakpoints_sal (struct gdbarch *,
 				    gdb::unique_xmalloc_ptr<char>,
 				    gdb::unique_xmalloc_ptr<char>,
 				    enum bptype,
-				    enum bpdisp, int, int,
+				    enum bpdisp, int, int, int,
 				    int,
 				    int, int, int, unsigned);
 
@@ -1462,11 +1462,28 @@  breakpoint_set_thread (struct breakpoint *b, int thread)
 {
   int old_thread = b->thread;
 
+  gdb_assert (thread == -1 || b->inferior == -1);
+
   b->thread = thread;
   if (old_thread != thread)
     gdb::observers::breakpoint_modified.notify (b);
 }
 
+/* Set the inferior for breakpoint B to INFERIOR.  If INFERIOR is -1, make
+   the breakpoint work for any inferior.  */
+
+void
+breakpoint_set_inferior (struct breakpoint *b, int inferior)
+{
+  int old_inferior = b->inferior;
+
+  gdb_assert (inferior == -1 || b->thread == -1);
+
+  b->inferior = inferior;
+  if (old_inferior != inferior)
+    gdb::observers::breakpoint_modified.notify (b);
+}
+
 /* Set the task for this breakpoint.  If TASK is 0, make the
    breakpoint work for any task.  */
 
@@ -3151,6 +3168,12 @@  insert_breakpoint_locations (void)
 	  && !valid_global_thread_id (bl->owner->thread))
 	continue;
 
+      /* Or inferior specific breakpoints if the inferior no longer
+	 exists.  */
+      if (bl->owner->inferior != -1
+	  && !valid_global_inferior_id (bl->owner->inferior))
+	continue;
+
       switch_to_program_space_and_thread (bl->pspace);
 
       /* For targets that support global breakpoints, there's no need
@@ -3255,6 +3278,32 @@  Thread-specific breakpoint %d deleted - thread %s no longer in the thread list.\
     }
 }
 
+/* Called when inferior INF has exited.  Remove per-inferior breakpoints.  */
+
+static void
+remove_inferior_breakpoints (struct inferior *inf)
+{
+  for (breakpoint *b : all_breakpoints_safe ())
+    {
+      if (b->inferior == inf->num && user_breakpoint_p (b))
+	{
+	  /* Tell the user the breakpoint has been deleted.  But only for
+	     breakpoints that would not normally have been deleted at the
+	     next stop anyway.  */
+	  if (b->disposition != disp_del
+	      && b->disposition != disp_del_at_next_stop)
+	    gdb_printf (_("\
+Inferior-specific breakpoint %d deleted - inferior %d has exited.\n"),
+			b->number, inf->num);
+
+
+	  /* Hide it from the user and mark it for deletion.  */
+	  b->number = 0;
+	  b->disposition = disp_del_at_next_stop;
+       }
+    }
+}
+
 /* See breakpoint.h.  */
 
 void
@@ -5464,6 +5513,7 @@  bpstat_check_breakpoint_conditions (bpstat *bs, thread_info *thread)
      evaluating the condition if this isn't the specified
      thread/task.  */
   if ((b->thread != -1 && b->thread != thread->global_num)
+      || (b->inferior != -1 && b->inferior != thread->inf->num)
       || (b->task != 0 && b->task != ada_get_task_number (thread)))
     {
       infrun_debug_printf ("incorrect thread or task, not stopping");
@@ -6481,6 +6531,11 @@  print_one_breakpoint_location (struct breakpoint *b,
 	  uiout->text (" task ");
 	  uiout->field_signed ("task", b->task);
 	}
+      else if (b->inferior != -1)
+	{
+	  uiout->text (" inferior ");
+	  uiout->field_signed ("inferior", b->inferior);
+	}
     }
 
   uiout->text ("\n");
@@ -6535,7 +6590,14 @@  print_one_breakpoint_location (struct breakpoint *b,
 	}
       uiout->text ("\n");
     }
-  
+
+  if (!part_of_multiple && b->inferior != -1)
+    {
+      uiout->text ("\tstop only in inferior ");
+      uiout->field_signed ("inferior", b->inferior);
+      uiout->text ("\n");
+    }
+
   if (!part_of_multiple)
     {
       if (b->hit_count)
@@ -7508,7 +7570,10 @@  delete_longjmp_breakpoint (int thread)
     if (b->type == bp_longjmp || b->type == bp_exception)
       {
 	if (b->thread == thread)
-	  delete_breakpoint (b);
+	  {
+	    gdb_assert (b->inferior == -1);
+	    delete_breakpoint (b);
+	  }
       }
 }
 
@@ -7519,7 +7584,10 @@  delete_longjmp_breakpoint_at_next_stop (int thread)
     if (b->type == bp_longjmp || b->type == bp_exception)
       {
 	if (b->thread == thread)
-	  b->disposition = disp_del_at_next_stop;
+	  {
+	    gdb_assert (b->inferior == -1);
+	    b->disposition = disp_del_at_next_stop;
+	  }
       }
 }
 
@@ -7572,6 +7640,7 @@  check_longjmp_breakpoint_for_call_dummy (struct thread_info *tp)
   ALL_BREAKPOINTS_SAFE (b, b_tmp)
     if (b->type == bp_longjmp_call_dummy && b->thread == tp->global_num)
       {
+	gdb_assert (b->inferior == -1);
 	struct breakpoint *dummy_b = b->related_breakpoint;
 
 	/* Find the bp_call_dummy breakpoint in the list of breakpoints
@@ -8404,7 +8473,8 @@  code_breakpoint::code_breakpoint (struct gdbarch *gdbarch_,
 				  gdb::unique_xmalloc_ptr<char> cond_string_,
 				  gdb::unique_xmalloc_ptr<char> extra_string_,
 				  enum bpdisp disposition_,
-				  int thread_, int task_, int ignore_count_,
+				  int thread_, int task_, int inferior_,
+				  int ignore_count_,
 				  int from_tty,
 				  int enabled_, unsigned flags,
 				  int display_canonical_)
@@ -8430,6 +8500,11 @@  code_breakpoint::code_breakpoint (struct gdbarch *gdbarch_,
 
   thread = thread_;
   task = task_;
+  inferior = inferior_;
+
+  /* A breakpoint can be thread specific, or inferior specific, but not
+     both.  This should be checked when the breakpoint condition is parsed.  */
+  gdb_assert (!(thread != -1 && inferior != -1));
 
   cond_string = std::move (cond_string_);
   extra_string = std::move (extra_string_);
@@ -8531,7 +8606,7 @@  create_breakpoint_sal (struct gdbarch *gdbarch,
 		       gdb::unique_xmalloc_ptr<char> cond_string,
 		       gdb::unique_xmalloc_ptr<char> extra_string,
 		       enum bptype type, enum bpdisp disposition,
-		       int thread, int task, int ignore_count,
+		       int thread, int task, int inferior, int ignore_count,
 		       int from_tty,
 		       int enabled, int internal, unsigned flags,
 		       int display_canonical)
@@ -8545,7 +8620,7 @@  create_breakpoint_sal (struct gdbarch *gdbarch,
 				std::move (cond_string),
 				std::move (extra_string),
 				disposition,
-				thread, task, ignore_count,
+				thread, task, inferior, ignore_count,
 				from_tty,
 				enabled, flags,
 				display_canonical);
@@ -8574,7 +8649,8 @@  create_breakpoints_sal (struct gdbarch *gdbarch,
 			gdb::unique_xmalloc_ptr<char> cond_string,
 			gdb::unique_xmalloc_ptr<char> extra_string,
 			enum bptype type, enum bpdisp disposition,
-			int thread, int task, int ignore_count,
+			int thread, int task, int inferior,
+			int ignore_count,
 			int from_tty,
 			int enabled, int internal, unsigned flags)
 {
@@ -8598,7 +8674,7 @@  create_breakpoints_sal (struct gdbarch *gdbarch,
 			     std::move (cond_string),
 			     std::move (extra_string),
 			     type, disposition,
-			     thread, task, ignore_count,
+			     thread, task, inferior, ignore_count,
 			     from_tty, enabled, internal, flags,
 			     canonical->special_display);
     }
@@ -8728,21 +8804,26 @@  check_fast_tracepoint_sals (struct gdbarch *gdbarch,
     }
 }
 
-/* Given TOK, a string specification of condition and thread, as
-   accepted by the 'break' command, extract the condition
-   string and thread number and set *COND_STRING and *THREAD.
-   PC identifies the context at which the condition should be parsed.
-   If no condition is found, *COND_STRING is set to NULL.
-   If no thread is found, *THREAD is set to -1.  */
+/* Given TOK, a string specification of condition and thread, as accepted
+   by the 'break' command, extract the condition string into *COND_STRING.
+   If no condition string is found then *COND_STRING is set to nullptr.
+
+   If the breakpoint specification has an associated thread, task, or
+   inferior, these are extracted into *THREAD, *TASK, and *INFERIOR
+   respectively, otherwise these arguments are set to -1 (for THREAD and
+   INFERIOR) or 0 (for TASK).
+
+   PC identifies the context at which the condition should be parsed.  */
 
 static void
 find_condition_and_thread (const char *tok, CORE_ADDR pc,
 			   gdb::unique_xmalloc_ptr<char> *cond_string,
-			   int *thread, int *task,
+			   int *thread, int *inferior, int *task,
 			   gdb::unique_xmalloc_ptr<char> *rest)
 {
   cond_string->reset ();
   *thread = -1;
+  *inferior = -1;
   *task = 0;
   rest->reset ();
   bool force = false;
@@ -8759,7 +8840,7 @@  find_condition_and_thread (const char *tok, CORE_ADDR pc,
       if ((*tok == '"' || *tok == ',') && rest)
 	{
 	  rest->reset (savestring (tok, strlen (tok)));
-	  return;
+	  break;
 	}
 
       end_tok = skip_to_space (tok);
@@ -8803,6 +8884,18 @@  find_condition_and_thread (const char *tok, CORE_ADDR pc,
 	  *thread = thr->global_num;
 	  tok = tmptok;
 	}
+      else if (toklen >= 1 && strncmp (tok, "inferior", toklen) == 0)
+	{
+	  char *tmptok;
+
+	  tok = end_tok + 1;
+	  *inferior = strtol (tok, &tmptok, 0);
+	  if (tok == tmptok)
+	    error (_("Junk after inferior keyword."));
+	  if (!valid_global_inferior_id (*inferior))
+	    error (_("Unknown inferior number %d."), *inferior);
+	  tok = tmptok;
+	}
       else if (toklen >= 1 && strncmp (tok, "task", toklen) == 0)
 	{
 	  char *tmptok;
@@ -8821,11 +8914,15 @@  find_condition_and_thread (const char *tok, CORE_ADDR pc,
       else if (rest)
 	{
 	  rest->reset (savestring (tok, strlen (tok)));
-	  return;
+	  break;
 	}
       else
 	error (_("Junk at end of arguments."));
     }
+
+  if (*thread != -1 && *inferior != -1)
+    error (_("Invalid use of both 'thread' and 'inferior' in "
+	     "breakpoint condition"));
 }
 
 /* Call 'find_condition_and_thread' for each sal in SALS until a parse
@@ -8837,7 +8934,7 @@  static void
 find_condition_and_thread_for_sals (const std::vector<symtab_and_line> &sals,
 				    const char *input,
 				    gdb::unique_xmalloc_ptr<char> *cond_string,
-				    int *thread, int *task,
+				    int *thread, int *inferior, int *task,
 				    gdb::unique_xmalloc_ptr<char> *rest)
 {
   int num_failures = 0;
@@ -8845,6 +8942,7 @@  find_condition_and_thread_for_sals (const std::vector<symtab_and_line> &sals,
     {
       gdb::unique_xmalloc_ptr<char> cond;
       int thread_id = 0;
+      int inferior_id = 0;
       int task_id = 0;
       gdb::unique_xmalloc_ptr<char> remaining;
 
@@ -8857,9 +8955,10 @@  find_condition_and_thread_for_sals (const std::vector<symtab_and_line> &sals,
       try
 	{
 	  find_condition_and_thread (input, sal.pc, &cond, &thread_id,
-				     &task_id, &remaining);
+				     &inferior_id, &task_id, &remaining);
 	  *cond_string = std::move (cond);
 	  *thread = thread_id;
+	  *inferior = inferior_id;
 	  *task = task_id;
 	  *rest = std::move (remaining);
 	  break;
@@ -8961,6 +9060,7 @@  create_breakpoint (struct gdbarch *gdbarch,
   struct linespec_result canonical;
   bool pending = false;
   int task = 0;
+  int inferior = -1;
   int prev_bkpt_count = breakpoint_count;
 
   gdb_assert (ops != NULL);
@@ -9038,7 +9138,8 @@  create_breakpoint (struct gdbarch *gdbarch,
 	  const linespec_sals &lsal = canonical.lsals[0];
 
 	  find_condition_and_thread_for_sals (lsal.sals, extra_string,
-					      &cond, &thread, &task, &rest);
+					      &cond, &thread, &inferior,
+					      &task, &rest);
 	  cond_string_copy = std::move (cond);
 	  extra_string_copy = std::move (rest);
 	}
@@ -9088,7 +9189,7 @@  create_breakpoint (struct gdbarch *gdbarch,
 				   std::move (extra_string_copy),
 				   type_wanted,
 				   tempflag ? disp_del : disp_donttouch,
-				   thread, task, ignore_count,
+				   thread, task, inferior, ignore_count,
 				   from_tty, enabled, internal, flags);
     }
   else
@@ -10028,6 +10129,7 @@  watch_command_1 (const char *arg, int accessflag, int from_tty,
   const char *cond_end = NULL;
   enum bptype bp_type;
   int thread = -1;
+  int inferior = -1;
   /* Flag to indicate whether we are going to use masks for
      the hardware watchpoint.  */
   bool use_mask = false;
@@ -10068,6 +10170,10 @@  watch_command_1 (const char *arg, int accessflag, int from_tty,
 	  tok++;
 	  toklen = end_tok - tok + 1;
 
+	  if (thread != -1 && inferior != -1)
+	    error (_("Invalid use of both 'thread' and 'inferior' in "
+		     "watchpoint condition"));
+
 	  if (toklen == 6 && startswith (tok, "thread"))
 	    {
 	      struct thread_info *thr;
@@ -10081,10 +10187,8 @@  watch_command_1 (const char *arg, int accessflag, int from_tty,
 
 	      /* Extract the thread ID from the next token.  */
 	      thr = parse_thread_id (value_start, &endp);
-
-	      /* Check if the user provided a valid thread ID.  */
-	      if (*endp != ' ' && *endp != '\t' && *endp != '\0')
-		invalid_thread_id_error (value_start);
+	      if (value_start == endp)
+		error (_("Junk after thread keyword."));
 
 	      thread = thr->global_num;
 	    }
@@ -10101,6 +10205,16 @@  watch_command_1 (const char *arg, int accessflag, int from_tty,
 	      if (!valid_task_id (task))
 		error (_("Unknown task %d."), task);
 	    }
+	  else if (toklen == 8 && startswith (tok, "inferior"))
+	    {
+	      char *tmp;
+
+	      inferior = strtol (value_start, &tmp, 0);
+	      if (tmp == value_start)
+		error (_("Junk after inferior keyword."));
+	      if (!valid_global_inferior_id (inferior))
+		error (_("Unknown inferior number %d."), inferior);
+	    }
 	  else if (toklen == 4 && startswith (tok, "mask"))
 	    {
 	      /* We've found a "mask" token, which means the user wants to
@@ -10271,6 +10385,7 @@  watch_command_1 (const char *arg, int accessflag, int from_tty,
     w.reset (new watchpoint (nullptr, bp_type));
 
   w->thread = thread;
+  w->inferior = inferior;
   w->task = task;
   w->disposition = disp_donttouch;
   w->pspace = current_program_space;
@@ -12182,7 +12297,8 @@  strace_marker_create_breakpoints_sal (struct gdbarch *gdbarch,
 				      enum bptype type_wanted,
 				      enum bpdisp disposition,
 				      int thread,
-				      int task, int ignore_count,
+				      int task, int inferior,
+				      int ignore_count,
 				      int from_tty, int enabled,
 				      int internal, unsigned flags)
 {
@@ -12208,7 +12324,7 @@  strace_marker_create_breakpoints_sal (struct gdbarch *gdbarch,
 			 std::move (cond_string),
 			 std::move (extra_string),
 			 disposition,
-			 thread, task, ignore_count,
+			 thread, task, inferior, ignore_count,
 			 from_tty, enabled, flags,
 			 canonical->special_display));
 
@@ -12841,10 +12957,11 @@  code_breakpoint::location_spec_to_sals (location_spec *locspec,
       if (condition_not_parsed && extra_string != NULL)
 	{
 	  gdb::unique_xmalloc_ptr<char> local_cond, local_extra;
-	  int local_thread, local_task;
+	  int local_thread, local_task, local_inferior;
 
 	  find_condition_and_thread_for_sals (sals, extra_string.get (),
 					      &local_cond, &local_thread,
+					      &local_inferior,
 					      &local_task, &local_extra);
 	  gdb_assert (cond_string == nullptr);
 	  if (local_cond != nullptr)
@@ -14930,4 +15047,6 @@  This is useful for formatted output in user-defined commands."));
 					   "breakpoint");
   gdb::observers::thread_exit.attach (remove_threaded_breakpoints,
 				      "breakpoint");
+  gdb::observers::inferior_exit.attach (remove_inferior_breakpoints,
+					"breakpoint");
 }
diff --git a/gdb/breakpoint.h b/gdb/breakpoint.h
index f7633d29cbf..7ee914ef1bd 100644
--- a/gdb/breakpoint.h
+++ b/gdb/breakpoint.h
@@ -584,7 +584,7 @@  struct breakpoint_ops
 				  struct linespec_result *,
 				  gdb::unique_xmalloc_ptr<char>,
 				  gdb::unique_xmalloc_ptr<char>,
-				  enum bptype, enum bpdisp, int, int,
+				  enum bptype, enum bpdisp, int, int, int,
 				  int, int, int, int, unsigned);
 };
 
@@ -801,6 +801,10 @@  struct breakpoint
      care.  */
   int thread = -1;
 
+  /* Inferior number for inferior-specific breakpoint, or -1 if this
+     breakpoint is for all inferiors.  */
+  int inferior = -1;
+
   /* Ada task number for task-specific breakpoint, or 0 if don't
      care.  */
   int task = 0;
@@ -856,7 +860,7 @@  struct code_breakpoint : public breakpoint
 		   gdb::unique_xmalloc_ptr<char> cond_string,
 		   gdb::unique_xmalloc_ptr<char> extra_string,
 		   enum bpdisp disposition,
-		   int thread, int task, int ignore_count,
+		   int thread, int task, int inferior, int ignore_count,
 		   int from_tty,
 		   int enabled, unsigned flags,
 		   int display_canonical);
@@ -1668,6 +1672,8 @@  extern void breakpoint_set_silent (struct breakpoint *b, int silent);
 
 extern void breakpoint_set_thread (struct breakpoint *b, int thread);
 
+extern void breakpoint_set_inferior (struct breakpoint *b, int inferior);
+
 extern void breakpoint_set_task (struct breakpoint *b, int task);
 
 /* Clear the "inserted" flag in all breakpoints.  */
diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
index bababf3c7ff..5cf6c43393f 100644
--- a/gdb/doc/gdb.texinfo
+++ b/gdb/doc/gdb.texinfo
@@ -3500,6 +3500,58 @@ 
 space as a result of inferior 1 having executed a @code{vfork} call.
 @end table
 
+@menu
+* Inferior-Specific Breakpoints::	Controlling breakpoints
+@end menu
+
+@node Inferior-Specific Breakpoints
+@subsection Inferior-Specific Breakpoints
+
+When debugging multiple inferiors, you can choose whether to set
+breakpoints for all inferiors, or for a particular inferior.
+
+@table @code
+@cindex breakpoints and inferiors
+@cindex inferior breakpoints
+@kindex break @dots{} inferior @var{inferior-id}
+@item break @var{locspec} inferior @var{inferior-id}
+@itemx break @var{locspec} inferior @var{inferior-id} if @dots{}
+@var{locspec} specifies a code location or locations in your program.
+@xref{Location Specifications}, for details.
+
+Use the qualifier @samp{inferior @var{inferior-id}} with a breakpoint
+command to specify that you only want @value{GDBN} to stop when a
+particular inferior reaches this breakpoint.  The @var{inferior-id}
+specifier is one of the inferior identifiers assigned by @value{GDBN},
+shown in the first column of the @samp{info inferiors} output.
+
+If you do not specify @samp{inferior @var{inferior-id}} when you set a
+breakpoint, the breakpoint applies to @emph{all} inferiors of your
+program.
+
+You can use the @code{inferior} qualifier on conditional breakpoints as
+well; in this case, place @samp{inferior @var{inferior-id}} before or
+after the breakpoint condition, like this:
+
+@smallexample
+(@value{GDBP}) break frik.c:13 inferior 2 if bartab > lim
+@end smallexample
+@end table
+
+Inferior-specific breakpoints are automatically deleted when
+@value{GDBN} detects the corresponding inferior has exited.  For
+example:
+
+@smallexample
+(@value{GDBP}) c
+Inferior-specific breakpoint 3 deleted - inferior 2 has exited.
+@end smallexample
+
+A breakpoint can't be both inferior-specific and thread-specific
+(@pxref{Thread-Specific Breakpoints}) using both the @code{inferior}
+and @code{thread} keywords when creating a breakpoint will give an
+error.
+
 @node Threads
 @section Debugging Programs with Multiple Threads
 
@@ -4462,8 +4514,9 @@ 
 situation.
 
 It is also possible to insert a breakpoint that will stop the program
-only if a specific thread (@pxref{Thread-Specific Breakpoints})
-or a specific task (@pxref{Ada Tasks}) hits that breakpoint.
+only if a specific thread (@pxref{Thread-Specific Breakpoints}),
+specific inferior (@pxref{Inferior-Specific Breakpoints}), or a
+specific task (@pxref{Ada Tasks}) hits that breakpoint.
 
 @item break
 When called without any arguments, @code{break} sets a breakpoint at
@@ -4979,7 +5032,7 @@ 
 
 @table @code
 @kindex watch
-@item watch @r{[}-l@r{|}-location@r{]} @var{expr} @r{[}thread @var{thread-id}@r{]} @r{[}mask @var{maskvalue}@r{]} @r{[}task @var{task-id}@r{]}
+@item watch @r{[}-l@r{|}-location@r{]} @var{expr} @r{[}thread @var{thread-id}@r{]} @r{[}inferior @var{inferior-id}@r{]} @r{[}mask @var{maskvalue}@r{]} @r{[}task @var{task-id}@r{]}
 Set a watchpoint for an expression.  @value{GDBN} will break when the
 expression @var{expr} is written into by the program and its value
 changes.  The simplest (and the most popular) use of this command is
@@ -4996,8 +5049,10 @@ 
 that watchpoints restricted to a single thread in this way only work
 with Hardware Watchpoints.
 
-Similarly, if the @code{task} argument is given, then the watchpoint
-will be specific to the indicated Ada task (@pxref{Ada Tasks}).
+Similarly, if the @code{inferior} argument is given, then the
+watchpoint will trigger only for the specific inferior, or if the
+@code{task} argument is given, then the watchpoint will be specific to
+the indicated Ada task (@pxref{Ada Tasks}).
 
 Ordinarily a watchpoint respects the scope of variables in @var{expr}
 (see below).  The @code{-location} argument tells @value{GDBN} to
@@ -5026,12 +5081,12 @@ 
 @end smallexample
 
 @kindex rwatch
-@item rwatch @r{[}-l@r{|}-location@r{]} @var{expr} @r{[}thread @var{thread-id}@r{]} @r{[}mask @var{maskvalue}@r{]}
+@item rwatch @r{[}-l@r{|}-location@r{]} @var{expr} @r{[}thread @var{thread-id}@r{]} @r{[}inferior @var{inferior-id}@r{]} @r{[}mask @var{maskvalue}@r{]}
 Set a watchpoint that will break when the value of @var{expr} is read
 by the program.
 
 @kindex awatch
-@item awatch @r{[}-l@r{|}-location@r{]} @var{expr} @r{[}thread @var{thread-id}@r{]} @r{[}mask @var{maskvalue}@r{]}
+@item awatch @r{[}-l@r{|}-location@r{]} @var{expr} @r{[}thread @var{thread-id}@r{]} @r{[}inferior @var{inferior-id}@r{]} @r{[}mask @var{maskvalue}@r{]}
 Set a watchpoint that will break when @var{expr} is either read from
 or written into by the program.
 
@@ -7313,6 +7368,11 @@ 
 explictly asks for the thread list with the @code{info threads}
 command.
 
+A breakpoint can't be both thread-specific and inferior-specific
+(@pxref{Inferior-Specific Breakpoints}) using both the @code{thread}
+and @code{inferior} keywords when creating a breakpoint will give an
+error.
+
 @node Interrupted System Calls
 @subsection Interrupted System Calls 
 
diff --git a/gdb/doc/python.texi b/gdb/doc/python.texi
index 9cbb2f9f57d..396d0f7f7af 100644
--- a/gdb/doc/python.texi
+++ b/gdb/doc/python.texi
@@ -3281,7 +3281,9 @@ 
 A @code{gdb.Inferior} object has the following attributes:
 
 @defvar Inferior.num
-ID of inferior, as assigned by GDB.
+ID of inferior, as assigned by GDB.  You can use this to
+make Python breakpoints inferior-specific, for example
+(@pxref{python_breakpoint_inferior,,The Breakpoint.inferior attribute}).
 @end defvar
 
 @anchor{gdbpy_inferior_connection}
@@ -6077,9 +6079,26 @@ 
 
 @anchor{python_breakpoint_thread}
 @defvar Breakpoint.thread
-If the breakpoint is thread-specific, this attribute holds the
-thread's global id.  If the breakpoint is not thread-specific, this
-attribute is @code{None}.  This attribute is writable.
+If the breakpoint is thread-specific (@pxref{Thread-Specific
+Breakpoints}), this attribute holds the thread's global id.  If the
+breakpoint is not thread-specific, this attribute is @code{None}.
+This attribute is writable.
+
+Only one of @code{Breakpoint.thread} or @code{Breakpoint.inferior} can
+be set to a valid id at any time, that is, a breakpoint can be thread
+specific, or inferior specific, but not both.
+@end defvar
+
+@anchor{python_breakpoint_inferior}
+@defvar Breakpoint.inferior
+If the breakpoint is inferior-specific (@pxref{Inferior-Specific
+Breakpoints}), this attribute holds the inferior's id.  If the
+breakpoint is not inferior-specific, this attribute is @code{None}.
+This attribute is writable.
+
+Only one of @code{Breakpoint.thread} or @code{Breakpoint.inferior} can
+be set to a valid id at any time, that is, a breakpoint can be thread
+specific, or inferior specific, but not both.
 @end defvar
 
 @defvar Breakpoint.task
diff --git a/gdb/dummy-frame.c b/gdb/dummy-frame.c
index 42c4bf23431..75ebf940a0c 100644
--- a/gdb/dummy-frame.c
+++ b/gdb/dummy-frame.c
@@ -132,6 +132,7 @@  pop_dummy_frame_bpt (struct breakpoint *b, struct dummy_frame *dummy)
   if (b->thread == dummy->id.thread->global_num
       && b->disposition == disp_del && b->frame_id == dummy->id.id)
     {
+      gdb_assert (b->inferior == -1);
       while (b->related_breakpoint != b)
 	delete_breakpoint (b->related_breakpoint);
 
diff --git a/gdb/elfread.c b/gdb/elfread.c
index 64aeb239670..ed3cad0cd02 100644
--- a/gdb/elfread.c
+++ b/gdb/elfread.c
@@ -968,7 +968,10 @@  elf_gnu_ifunc_resolver_stop (code_breakpoint *b)
       if (b_return->thread == thread_id
 	  && b_return->loc->requested_address == prev_pc
 	  && b_return->frame_id == prev_frame_id)
-	break;
+	{
+	  gdb_assert (b_return->inferior == -1);
+	  break;
+	}
     }
 
   if (b_return == b)
diff --git a/gdb/guile/scm-breakpoint.c b/gdb/guile/scm-breakpoint.c
index a7799416c50..90b66a7af86 100644
--- a/gdb/guile/scm-breakpoint.c
+++ b/gdb/guile/scm-breakpoint.c
@@ -779,6 +779,11 @@  gdbscm_set_breakpoint_thread_x (SCM self, SCM newvalue)
   else
     SCM_ASSERT_TYPE (0, newvalue, SCM_ARG2, FUNC_NAME, _("integer or #f"));
 
+  if (bp_smob->bp->inferior != -1)
+    scm_misc_error (FUNC_NAME,
+		    _("Cannot have both thread and inferior conditions "
+		      "on a breakpoint"), SCM_EOL);
+
   breakpoint_set_thread (bp_smob->bp, id);
 
   return SCM_UNSPECIFIED;
diff --git a/gdb/inferior.h b/gdb/inferior.h
index 69525a2e053..3b25403d83c 100644
--- a/gdb/inferior.h
+++ b/gdb/inferior.h
@@ -746,4 +746,14 @@  extern void print_selected_inferior (struct ui_out *uiout);
 extern void switch_to_inferior_and_push_target
   (inferior *new_inf, bool no_connection, inferior *org_inf);
 
+/* Return true if ID is a valid global inferior number.  */
+
+inline bool valid_global_inferior_id (int id)
+{
+  for (inferior *inf : all_inferiors ())
+    if (inf->num == id)
+      return true;
+  return false;
+}
+
 #endif /* !defined (INFERIOR_H) */
diff --git a/gdb/infrun.c b/gdb/infrun.c
index 96346e1f25b..0650b3c1528 100644
--- a/gdb/infrun.c
+++ b/gdb/infrun.c
@@ -8039,6 +8039,7 @@  insert_exception_resume_breakpoint (struct thread_info *tp,
 	  frame = nullptr;
 
 	  bp->thread = tp->global_num;
+	  bp->inferior = -1;
 	  inferior_thread ()->control.exception_resume_breakpoint = bp;
 	}
     }
@@ -8072,6 +8073,7 @@  insert_exception_resume_from_probe (struct thread_info *tp,
   bp = set_momentary_breakpoint_at_pc (get_frame_arch (frame),
 				       handler, bp_exception_resume).release ();
   bp->thread = tp->global_num;
+  bp->inferior = -1;
   inferior_thread ()->control.exception_resume_breakpoint = bp;
 }
 
diff --git a/gdb/linespec.c b/gdb/linespec.c
index 3db35998f7e..b01423107f9 100644
--- a/gdb/linespec.c
+++ b/gdb/linespec.c
@@ -254,9 +254,9 @@  enum linespec_token_type
 
 /* List of keywords.  This is NULL-terminated so that it can be used
    as enum completer.  */
-const char * const linespec_keywords[] = { "if", "thread", "task", "-force-condition", NULL };
+const char * const linespec_keywords[] = { "if", "thread", "task", "inferior", "-force-condition", NULL };
 #define IF_KEYWORD_INDEX 0
-#define FORCE_KEYWORD_INDEX 3
+#define FORCE_KEYWORD_INDEX 4
 
 /* A token of the linespec lexer  */
 
diff --git a/gdb/python/py-breakpoint.c b/gdb/python/py-breakpoint.c
index 7a757432948..31d447a468f 100644
--- a/gdb/python/py-breakpoint.c
+++ b/gdb/python/py-breakpoint.c
@@ -280,11 +280,69 @@  bppy_set_thread (PyObject *self, PyObject *newvalue, void *closure)
       return -1;
     }
 
+  if (self_bp->bp->inferior != -1 && id != -1)
+    {
+      PyErr_SetString (PyExc_RuntimeError,
+		       _("Cannot have both thread and inferior conditions "
+			 "on a breakpoint"));
+      return -1;
+    }
+
   breakpoint_set_thread (self_bp->bp, id);
 
   return 0;
 }
 
+/* Python function to set the inferior of a breakpoint.  */
+
+static int
+bppy_set_inferior (PyObject *self, PyObject *newvalue, void *closure)
+{
+  gdbpy_breakpoint_object *self_bp = (gdbpy_breakpoint_object *) self;
+  long id;
+
+  BPPY_SET_REQUIRE_VALID (self_bp);
+
+  if (newvalue == NULL)
+    {
+      PyErr_SetString (PyExc_TypeError,
+		       _("Cannot delete `inferior' attribute."));
+      return -1;
+    }
+  else if (PyLong_Check (newvalue))
+    {
+      if (!gdb_py_int_as_long (newvalue, &id))
+	return -1;
+
+      if (!valid_global_inferior_id (id))
+	{
+	  PyErr_SetString (PyExc_RuntimeError,
+			   _("Invalid inferior ID."));
+	  return -1;
+	}
+    }
+  else if (newvalue == Py_None)
+    id = -1;
+  else
+    {
+      PyErr_SetString (PyExc_TypeError,
+		       _("The value of `inferior' must be an integer or None."));
+      return -1;
+    }
+
+  if (self_bp->bp->thread != -1 && id != -1)
+    {
+      PyErr_SetString (PyExc_RuntimeError,
+		       _("Cannot have both thread and inferior conditions "
+			 "on a breakpoint"));
+      return -1;
+    }
+
+  breakpoint_set_inferior (self_bp->bp, id);
+
+  return 0;
+}
+
 /* Python function to set the (Ada) task of a breakpoint.  */
 static int
 bppy_set_task (PyObject *self, PyObject *newvalue, void *closure)
@@ -689,6 +747,20 @@  bppy_get_thread (PyObject *self, void *closure)
   return gdb_py_object_from_longest (self_bp->bp->thread).release ();
 }
 
+/* Python function to get the breakpoint's inferior ID.  */
+static PyObject *
+bppy_get_inferior (PyObject *self, void *closure)
+{
+  gdbpy_breakpoint_object *self_bp = (gdbpy_breakpoint_object *) self;
+
+  BPPY_REQUIRE_VALID (self_bp);
+
+  if (self_bp->bp->inferior == -1)
+    Py_RETURN_NONE;
+
+  return gdb_py_object_from_longest (self_bp->bp->inferior).release ();
+}
+
 /* Python function to get the breakpoint's task ID (in Ada).  */
 static PyObject *
 bppy_get_task (PyObject *self, void *closure)
@@ -1335,6 +1407,11 @@  static gdb_PyGetSetDef breakpoint_object_getset[] = {
 If the value is a thread ID (integer), then this is a thread-specific breakpoint.\n\
 If the value is None, then this breakpoint is not thread-specific.\n\
 No other type of value can be used.", NULL },
+  { "inferior", bppy_get_inferior, bppy_set_inferior,
+    "Inferior ID for the breakpoint.\n\
+If the value is an inferior ID (integer), then this is an inferior-specific\n\
+breakpoint.  If the value is None, then this breakpoint is not\n\
+inferior-specific.  No other type of value can be used.", NULL },
   { "task", bppy_get_task, bppy_set_task,
     "Thread ID for the breakpoint.\n\
 If the value is a task ID (integer), then this is an Ada task-specific breakpoint.\n\
diff --git a/gdb/testsuite/gdb.linespec/cpcompletion.exp b/gdb/testsuite/gdb.linespec/cpcompletion.exp
index 0383469828c..650d0e75ea3 100644
--- a/gdb/testsuite/gdb.linespec/cpcompletion.exp
+++ b/gdb/testsuite/gdb.linespec/cpcompletion.exp
@@ -1259,8 +1259,8 @@  proc_with_prefix function-labels {} {
 }
 
 # Test that completion after a function name offers keyword
-# (if/task/thread/-force-condition) matches in linespec mode, and also
-# the explicit location options in explicit locations mode.
+# (if/inferior/task/thread/-force-condition) matches in linespec mode,
+# and also the explicit location options in explicit locations mode.
 
 proc_with_prefix keywords-after-function {} {
     set explicit_list \
diff --git a/gdb/testsuite/gdb.linespec/explicit.exp b/gdb/testsuite/gdb.linespec/explicit.exp
index 9064c137e13..2e4c42f3aba 100644
--- a/gdb/testsuite/gdb.linespec/explicit.exp
+++ b/gdb/testsuite/gdb.linespec/explicit.exp
@@ -412,6 +412,7 @@  namespace eval $testfile {
 	    "-qualified"
 	    "-source"
 	    "if"
+	    "inferior"
 	    "task"
 	    "thread"
 	}
diff --git a/gdb/testsuite/gdb.multi/inferior-specific-bp-1.c b/gdb/testsuite/gdb.multi/inferior-specific-bp-1.c
new file mode 100644
index 00000000000..26cb01911d2
--- /dev/null
+++ b/gdb/testsuite/gdb.multi/inferior-specific-bp-1.c
@@ -0,0 +1,52 @@ 
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2022 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+volatile int global_var = 0;
+
+static void
+stop_breakpt (void)
+{
+  /* Nothing.  */
+}
+
+static inline void __attribute__((__always_inline__))
+foo (void)
+{
+  int i;
+
+  for (i = 0; i < 10; ++i)
+    global_var = 0;
+}
+
+static void
+bar (void)
+{
+  global_var = 0;
+
+  foo ();
+}
+
+
+int
+main (void)
+{
+  global_var = 0;
+  foo ();
+  bar ();
+  stop_breakpt ();
+  return 0;
+}
diff --git a/gdb/testsuite/gdb.multi/inferior-specific-bp-2.c b/gdb/testsuite/gdb.multi/inferior-specific-bp-2.c
new file mode 100644
index 00000000000..3ddb1a17446
--- /dev/null
+++ b/gdb/testsuite/gdb.multi/inferior-specific-bp-2.c
@@ -0,0 +1,52 @@ 
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2022 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+static int bar (void);
+static int baz (void);
+static int foo (void);
+
+static void
+stop_breakpt (void)
+{
+  /* Nothing.  */
+}
+
+int
+main (void)
+{
+  int ret = baz ();
+  stop_breakpt ();
+  return ret;
+}
+
+static int
+bar (void)
+{
+  return baz ();
+}
+
+static int
+foo (void)
+{
+  return 0;
+}
+
+static int
+baz (void)
+{
+  return foo ();
+}
diff --git a/gdb/testsuite/gdb.multi/inferior-specific-bp.exp b/gdb/testsuite/gdb.multi/inferior-specific-bp.exp
new file mode 100644
index 00000000000..5d65f19b88c
--- /dev/null
+++ b/gdb/testsuite/gdb.multi/inferior-specific-bp.exp
@@ -0,0 +1,183 @@ 
+# Copyright 2022 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Test inferior-specific breakpoints.
+
+standard_testfile -1.c -2.c
+
+if { [use_gdb_stub] } {
+    return
+}
+
+set srcfile1 ${srcfile}
+set binfile1 ${binfile}-1
+set binfile2 ${binfile}-2
+
+if { [build_executable ${testfile}.exp ${binfile1} "${srcfile1}"] != 0 } {
+    return -1
+}
+
+if { [build_executable ${testfile}.exp ${binfile2} "${srcfile2}"] != 0 } {
+    return -1
+}
+
+# Start the first inferior.
+clean_restart ${binfile1}
+if {![runto_main]} {
+    return
+}
+
+# Add a second inferior, and start this one too.
+gdb_test "add-inferior" "Added inferior 2.*" "add empty inferior 2"
+gdb_test "inferior 2" "Switching to inferior 2.*" "switch to inferior 2"
+gdb_load $binfile2
+if {![runto_main]} {
+    return
+}
+
+# Try to create a breakpoint using both the 'inferior' and 'thread' keywords,
+# this should fail.  Try with the keywords in both orders just in case the
+# parser has a bug.
+gdb_test "break foo thread 1.1 inferior 1" \
+    "Invalid use of both 'thread' and 'inferior' in breakpoint condition"
+gdb_test "break foo inferior 1 thread 1.1" \
+    "Invalid use of both 'thread' and 'inferior' in breakpoint condition"
+
+# While w're here, check that we can't create a watchpoint with both thread
+# and inferior keywords.  As above, test with keywords in both orders.
+foreach type {watch rwatch awatch} {
+    gdb_test "$type global_var thread 1.1 inferior 1" \
+	"Invalid use of both 'thread' and 'inferior' in watchpoint condition"
+    gdb_test "$type global_var inferior 1 thread 1.1" \
+	"Invalid use of both 'thread' and 'inferior' in watchpoint condition"
+}
+
+# Clear out any other breakpoints.
+gdb_test "with confirm off -- delete breakpoints"
+
+# Create an inferior specific breakpoint.
+gdb_test "break foo inferior 1" \
+    "Breakpoint $decimal at $hex: foo\\. \\(3 locations\\)"
+
+set saw_header false
+set location_count 0
+set saw_inf_cond false
+gdb_test_multiple "info breakpoints" "" {
+    -re "^info breakpoints\r\n" {
+	exp_continue
+    }
+
+    -re "^Num\\s+\[^\r\n\]+\r\n" {
+	exp_continue
+    }
+
+    -re "^$decimal\\s+breakpoint\\s+keep\\s+y\\s+<MULTIPLE>\\s*\r\n" {
+	set saw_header true
+	exp_continue
+    }
+
+    -re "^\\s+stop only in inferior 1\r\n" {
+	set saw_inf_cond true
+	exp_continue
+    }
+
+    -re "^$decimal\\.\[123\]\\s+y\\s+ $hex in foo at \[^\r\n\]+ inf \[12\] inferior 1\r\n" {
+	incr location_count
+	exp_continue
+    }
+
+    -re "^$gdb_prompt $" {
+	with_test_prefix $gdb_test_name {
+	    gdb_assert { $saw_header }
+	    gdb_assert { $location_count == 3 }
+	    gdb_assert { $saw_inf_cond }
+	}
+    }
+}
+
+# Create a multi-inferior breakpoint to stop at.
+gdb_test "break stop_breakpt"
+
+# Now resume inferior 2, this should reach 'stop_breakpt'.
+gdb_test "continue" "hit Breakpoint $decimal\.$decimal, stop_breakpt \\(\\) .*" \
+    "continue in inferior 2"
+
+# Switch to inferior 1, and try there.
+gdb_test "inferior 1" ".*" \
+    "select inferior 1 to check the inferior-specific b/p works"
+gdb_test "continue " "hit Breakpoint $decimal\.$decimal, foo \\(\\) .*" \
+    "first continue in inferior 1"
+
+# Now back to inferior 2, let the inferior exit, the inferior-specific
+# breakpoint should not be deleted.
+gdb_test "inferior 2" ".*" \
+    "switch back to allow inferior 2 to exit"
+gdb_test "continue" "Inferior 2 \[^\r\n\]+ exited normally.*" \
+    "allow inferior 2 to exit"
+
+gdb_test "inferior 1" ".*" \
+    "select inferior 1 to check inferior-specific b/p still works"
+gdb_test "continue " "hit Breakpoint $decimal\.$decimal, foo \\(\\) .*" \
+    "second continue in inferior 1"
+gdb_test "continue " "hit Breakpoint $decimal\.$decimal, stop_breakpt \\(\\) .*" \
+    "third continue in inferior 1"
+
+# Now allow inferior 1 to exit, the inferior specific breakpoint should be
+# deleted.
+gdb_test "continue" \
+    [multi_line \
+	 "\\\[Inferior 1 \[^\r\n\]+ exited normally\\\]" \
+	 "Inferior-specific breakpoint $decimal deleted - inferior 1 has exited\\."]
+
+set saw_header false
+set location_count 0
+set saw_inf_cond false
+gdb_test_multiple "info breakpoints" "info breakpoint after inferior 1 exited" {
+    -re "^info breakpoints\r\n" {
+	exp_continue
+    }
+
+    -re "^Num\\s+\[^\r\n\]+\r\n" {
+	exp_continue
+    }
+
+    -re "^$decimal\\s+breakpoint\\s+keep\\s+y\\s+<MULTIPLE>\\s*\r\n" {
+	set saw_header true
+	exp_continue
+    }
+
+    -re "^\\s+stop only in inferior 1\r\n" {
+	# This should not happen, this breakpoint should have been deleted.
+	set saw_inf_cond true
+	exp_continue
+    }
+
+    -re "^\\s+breakpoint already hit 2 times\r\n" {
+	exp_continue
+    }
+
+    -re "^$decimal\\.\[12\]\\s+y\\s+ $hex in stop_breakpt at \[^\r\n\]+ inf \[12\]\r\n" {
+	incr location_count
+	exp_continue
+    }
+
+    -re "^$gdb_prompt $" {
+	with_test_prefix $gdb_test_name {
+	    gdb_assert { $saw_header }
+	    gdb_assert { $location_count == 2 }
+	    gdb_assert { !$saw_inf_cond }
+	}
+    }
+}
diff --git a/gdb/testsuite/gdb.python/py-breakpoint.exp b/gdb/testsuite/gdb.python/py-breakpoint.exp
index c3215b13d5c..8906dea6655 100644
--- a/gdb/testsuite/gdb.python/py-breakpoint.exp
+++ b/gdb/testsuite/gdb.python/py-breakpoint.exp
@@ -112,6 +112,8 @@  proc_with_prefix test_bkpt_basic { } {
 	"Get Breakpoint List" 0
     gdb_test "python print (blist\[1\].thread)" \
 	"None" "Check breakpoint thread"
+    gdb_test "python print (blist\[1\].inferior)" \
+	"None" "Check breakpoint inferior"
     gdb_test "python print (blist\[1\].type == gdb.BP_BREAKPOINT)" \
 	"True" "Check breakpoint type"
     gdb_test "python print (blist\[0\].number)" \
@@ -214,6 +216,46 @@  proc_with_prefix test_bkpt_cond_and_cmds { } {
 	"check number of lines in commands"
 }
 
+# Test breakpoint thread and inferior attributes.
+proc_with_prefix test_bkpt_thread_and_inferior { } {
+    global srcfile testfile hex decimal
+
+    # Start with a fresh gdb.
+    clean_restart ${testfile}
+
+    if ![runto_main] then {
+	return 0
+    }
+
+    with_test_prefix "thread" {
+	delete_breakpoints
+	gdb_test "break multiply thread 1"
+	gdb_test "python bp = gdb.breakpoints ()\[0\]"
+	gdb_test "python print(bp.thread)" "1"
+	gdb_test "python print(bp.inferior)" "None"
+	gdb_test "python bp.inferior = 1" \
+	    "RuntimeError: Cannot have both thread and inferior conditions on a breakpoint.*"
+	gdb_test_no_output "python bp.thread = None"
+	gdb_test_no_output "python bp.inferior = 1" \
+	    "set the inferior now the thread has been cleared"
+	gdb_test "info breakpoints" "stop only in inferior 1\r\n.*"
+    }
+
+    with_test_prefix "inferior" {
+	delete_breakpoints
+	gdb_test "break multiply inferior 1"
+	gdb_test "python bp = gdb.breakpoints ()\[0\]"
+	gdb_test "python print(bp.thread)" "None"
+	gdb_test "python print(bp.inferior)" "1"
+	gdb_test "python bp.thread = 1" \
+	    "RuntimeError: Cannot have both thread and inferior conditions on a breakpoint.*"
+	gdb_test_no_output "python bp.inferior = None"
+	gdb_test_no_output "python bp.thread = 1" \
+	    "set the thread now the inferior has been cleared"
+	gdb_test "info breakpoints" "stop only in thread 1\r\n.*"
+    }
+}
+
 proc_with_prefix test_bkpt_invisible { } {
     global srcfile testfile hex decimal
 
@@ -849,6 +891,7 @@  proc_with_prefix test_bkpt_auto_disable { } {
 test_bkpt_basic
 test_bkpt_deletion
 test_bkpt_cond_and_cmds
+test_bkpt_thread_and_inferior
 test_bkpt_invisible
 test_hardware_breakpoints
 test_catchpoints
diff --git a/gdb/testsuite/lib/completion-support.exp b/gdb/testsuite/lib/completion-support.exp
index 0c03d0f035e..135649ed4d9 100644
--- a/gdb/testsuite/lib/completion-support.exp
+++ b/gdb/testsuite/lib/completion-support.exp
@@ -27,7 +27,7 @@  namespace eval completion {
     # List of all quote chars, including no-quote at all.
     variable maybe_quoted_list {"" "'" "\""}
 
-    variable keyword_list {"-force-condition" "if" "task" "thread"}
+    variable keyword_list {"-force-condition" "if" "inferior" "task" "thread"}
 
     variable explicit_opts_list \
 	{"-function" "-label" "-line" "-qualified" "-source"}