[RFA] Make "bt N" print correct number of frames when using a frame filter

  PR python/16497 notes that using "bt" with a positive argument prints
the wrong number of frames when a frame filter is in use.  Also, in this
case, the non-frame-filter path will print a message about "More stack
frames" when there are more; but this is not done in the frame-filter
case.

The first problem is that backtrace_command_1 passes the wrong value
to apply_ext_lang_frame_filter -- that function takes the final
frame's number as an argument, but backtrace_command_1 passes the
count, which is off by one.

The solution to the second problem is to have the C stack-printing
code stop at the correct number of frames and then print the message.

Tested using the buildbot.

ChangeLog
2017-04-22  Tom Tromey  <tom@tromey.com>

	PR python/16497:
	* stack.c (backtrace_command_1): Set PRINT_MORE_FRAMES flag.  Fix
	off-by-one in py_end computation.
	* python/py-framefilter.c (gdbpy_apply_frame_filter): Handle
	PRINT_MORE_FRAMES.
	* extension.h (enum frame_filter_flags) <PRINT_MORE_FRAMES>: New
	constant.

testsuite/ChangeLog
2017-04-22  Tom Tromey  <tom@tromey.com>

	PR python/16497:
	* gdb.python/py-framefilter.exp: Update test.
---
 gdb/ChangeLog                               | 10 ++++++++++
 gdb/extension.h                             |  3 +++
 gdb/python/py-framefilter.c                 | 25 +++++++++++++++++++++++++
 gdb/stack.c                                 |  6 +++++-
 gdb/testsuite/ChangeLog                     |  5 +++++
 gdb/testsuite/gdb.python/py-framefilter.exp |  2 +-
 6 files changed, 49 insertions(+), 2 deletions(-)
  

Thanks for the patch, Tom.

On Sunday, April 23 2017, Tom Tromey wrote:

> PR python/16497 notes that using "bt" with a positive argument prints
> the wrong number of frames when a frame filter is in use.  Also, in this
> case, the non-frame-filter path will print a message about "More stack
> frames" when there are more; but this is not done in the frame-filter
> case.
>
> The first problem is that backtrace_command_1 passes the wrong value
> to apply_ext_lang_frame_filter -- that function takes the final
> frame's number as an argument, but backtrace_command_1 passes the
> count, which is off by one.
>
> The solution to the second problem is to have the C stack-printing
> code stop at the correct number of frames and then print the message.
>
> Tested using the buildbot.
>
> ChangeLog
> 2017-04-22  Tom Tromey  <tom@tromey.com>
>
> 	PR python/16497:
> 	* stack.c (backtrace_command_1): Set PRINT_MORE_FRAMES flag.  Fix
> 	off-by-one in py_end computation.
> 	* python/py-framefilter.c (gdbpy_apply_frame_filter): Handle
> 	PRINT_MORE_FRAMES.
> 	* extension.h (enum frame_filter_flags) <PRINT_MORE_FRAMES>: New
> 	constant.
>
> testsuite/ChangeLog
> 2017-04-22  Tom Tromey  <tom@tromey.com>
>
> 	PR python/16497:
> 	* gdb.python/py-framefilter.exp: Update test.
> ---
>  gdb/ChangeLog                               | 10 ++++++++++
>  gdb/extension.h                             |  3 +++
>  gdb/python/py-framefilter.c                 | 25 +++++++++++++++++++++++++
>  gdb/stack.c                                 |  6 +++++-
>  gdb/testsuite/ChangeLog                     |  5 +++++
>  gdb/testsuite/gdb.python/py-framefilter.exp |  2 +-
>  6 files changed, 49 insertions(+), 2 deletions(-)
>
> diff --git a/gdb/ChangeLog b/gdb/ChangeLog
> index 28ae0a7..85d5da5 100644
> --- a/gdb/ChangeLog
> +++ b/gdb/ChangeLog
> @@ -1,5 +1,15 @@
>  2017-04-22  Tom Tromey  <tom@tromey.com>
>  
> +	PR python/16497:
> +	* stack.c (backtrace_command_1): Set PRINT_MORE_FRAMES flag.  Fix
> +	off-by-one in py_end computation.
> +	* python/py-framefilter.c (gdbpy_apply_frame_filter): Handle
> +	PRINT_MORE_FRAMES.
> +	* extension.h (enum frame_filter_flags) <PRINT_MORE_FRAMES>: New
> +	constant.
> +
> +2017-04-22  Tom Tromey  <tom@tromey.com>
> +
>  	* mi/mi-cmd-file.c (mi_cmd_file_list_shared_libraries): Use
>  	ui_out_emit_list.
>  	* stack.c (print_frame): Use ui_out_emit_list.
> diff --git a/gdb/extension.h b/gdb/extension.h
> index 2c79411..cda2ebf 100644
> --- a/gdb/extension.h
> +++ b/gdb/extension.h
> @@ -100,6 +100,9 @@ enum frame_filter_flags
>  
>      /* Set this flag if frame locals are to be printed.  */
>      PRINT_LOCALS = 8,
> +
> +    /* Set this flag if a "More frames" message is to be printed.  */
> +    PRINT_MORE_FRAMES = 16,
>    };

Not that I want you to fix this, but I like when bitflags are set using
the "1 << X" notation.  Oh, well...

>  
>  /* A choice of the different frame argument printing strategies that
> diff --git a/gdb/python/py-framefilter.c b/gdb/python/py-framefilter.c
> index 75b055c..b604d51 100644
> --- a/gdb/python/py-framefilter.c
> +++ b/gdb/python/py-framefilter.c
> @@ -1355,6 +1355,18 @@ gdbpy_apply_frame_filter (const struct extension_language_defn *extlang,
>  
>    gdbpy_enter enter_py (gdbarch, current_language);
>  
> +  /* When we're limiting the number of frames, be careful to request
> +     one extra frame, so that we can print a message if there are more
> +     frames.  */
> +  int frame_countdown = -1;
> +  if ((flags & PRINT_MORE_FRAMES) != 0 && frame_low >= 0 && frame_high >= 0)
> +    {
> +      ++frame_high;
> +      /* This has an extra +1 because it is checked before a frame is
> +	 printed.  */
> +      frame_countdown = frame_high - frame_low + 1;
> +    }
> +
>    gdbpy_ref<> iterable (bootstrap_python_frame_filters (frame, frame_low,
>  							frame_high));
>  
> @@ -1402,6 +1414,19 @@ gdbpy_apply_frame_filter (const struct extension_language_defn *extlang,
>  	  break;
>  	}
>  
> +      if (frame_countdown != -1)
> +	{
> +	  assert ((flags & PRINT_MORE_FRAMES) != 0);

gdb_assert here, right?

> +	  --frame_countdown;
> +	  if (frame_countdown == 0)
> +	    {
> +	      /* We've printed all the frames we were asked to
> +		 print, but more frames existed.  */
> +	      printf_filtered (_("(More stack frames follow...)\n"));
> +	      break;
> +	    }
> +	}
> +
>        success = py_print_frame (item.get (), flags, args_type, out, 0,
>  				levels_printed.get ());
>  
> diff --git a/gdb/stack.c b/gdb/stack.c
> index 7f8a51c..37e8767 100644
> --- a/gdb/stack.c
> +++ b/gdb/stack.c
> @@ -1766,7 +1766,9 @@ backtrace_command_1 (char *count_exp, int show_locals, int no_filters,
>        else
>  	{
>  	  py_start = 0;
> -	  py_end = count;
> +	  /* The argument to apply_ext_lang_frame_filter is the number
> +	     of the final frame to print, and frames start at 0.  */
> +	  py_end = count - 1;
>  	}
>      }
>    else
> @@ -1800,6 +1802,8 @@ backtrace_command_1 (char *count_exp, int show_locals, int no_filters,
>  
>        if (show_locals)
>  	flags |= PRINT_LOCALS;
> +      if (from_tty)
> +	flags |= PRINT_MORE_FRAMES;
>  
>        if (!strcmp (print_frame_arguments, "scalars"))
>  	arg_type = CLI_SCALAR_VALUES;
> diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog
> index c4d5b79..df883da 100644
> --- a/gdb/testsuite/ChangeLog
> +++ b/gdb/testsuite/ChangeLog
> @@ -1,3 +1,8 @@
> +2017-04-22  Tom Tromey  <tom@tromey.com>
> +
> +	PR python/16497:
> +	* gdb.python/py-framefilter.exp: Update test.
> +
>  2017-04-19  Pedro Alves  <palves@redhat.com>
>  
>  	* gdb.threads/threadapply.exp (kill_and_remove_inferior): New
> diff --git a/gdb/testsuite/gdb.python/py-framefilter.exp b/gdb/testsuite/gdb.python/py-framefilter.exp
> index bbec48d..d81d144 100644
> --- a/gdb/testsuite/gdb.python/py-framefilter.exp
> +++ b/gdb/testsuite/gdb.python/py-framefilter.exp
> @@ -149,7 +149,7 @@ gdb_test "bt -2" \
>      ".*#26.*func5.*#27.*in main \\(\\).*" \
>      "bt -2 with frame-filter Reverse disabled"
>  gdb_test "bt 3" \
> -    ".*#0.*end_func.*#1.*in funca \\(\\).*#2.*in funcb \\(j=10\\).*" \
> +    ".*#0.*end_func.*#1.*in funca \\(\\).*#2.*in funcb \\(j=10\\)\[^#\]*More stack frames follow.*" \
>      "bt 3 with frame-filter Reverse disabled"
>  gdb_test "bt no-filter full" \
>      ".*#0.*end_func.*str = $hex \"The End\".*st2 = $hex \"Is Near\".*b = 12.*c = 5.*#1.*in funca \\(\\).*#2.*in funcb \\(j=10\\).*bar = \{a = 42, b = 84\}.*" \
> -- 
> 2.9.3

Otherwise, LGTM.

Thanks,
  

>> +    /* Set this flag if a "More frames" message is to be printed.  */
>> +    PRINT_MORE_FRAMES = 16,
>> };

Sergio> Not that I want you to fix this, but I like when bitflags are set using
Sergio> the "1 << X" notation.  Oh, well...

I added a patch for this and I also changed this code to use
DEF_ENUM_FLAGS_TYPE, which seems better.  I'll send this change
separately.

>> +	  assert ((flags & PRINT_MORE_FRAMES) != 0);

Sergio> gdb_assert here, right?

Oops, yes.  I made this change.

Tom
  

> PR python/16497 notes that using "bt" with a positive argument prints
> the wrong number of frames when a frame filter is in use.

Going through my un-reviewed patches today.
Ping for this one.

Tom
  

On 23/04/17 17:04, Tom Tromey wrote:
> PR python/16497 notes that using "bt" with a positive argument prints
> the wrong number of frames when a frame filter is in use.  Also, in this
> case, the non-frame-filter path will print a message about "More stack
> frames" when there are more; but this is not done in the frame-filter
> case.
> 
> The first problem is that backtrace_command_1 passes the wrong value
> to apply_ext_lang_frame_filter -- that function takes the final
> frame's number as an argument, but backtrace_command_1 passes the
> count, which is off by one.
> 
> The solution to the second problem is to have the C stack-printing
> code stop at the correct number of frames and then print the message.

Good catch. Patch LGTM.

Cheers

Phil
  

On 04/23/2017 05:04 PM, Tom Tromey wrote:
> PR python/16497 notes that using "bt" with a positive argument prints
> the wrong number of frames when a frame filter is in use.  Also, in this
> case, the non-frame-filter path will print a message about "More stack
> frames" when there are more; but this is not done in the frame-filter
> case.
> 
> The first problem is that backtrace_command_1 passes the wrong value
> to apply_ext_lang_frame_filter -- that function takes the final
> frame's number as an argument, but backtrace_command_1 passes the
> count, which is off by one.
> 
> The solution to the second problem is to have the C stack-printing
> code stop at the correct number of frames and then print the message.
> 
> Tested using the buildbot.

Can you expand on the need for the PRINT_MORE_FRAMES flag and
having it based on from_tty?  I assume that your first made
the printing unconditional, but then for some reason decided
against it?

Thanks,
Pedro Alves
  

>>>>> "Pedro" == Pedro Alves <palves@redhat.com> writes:

Pedro> Can you expand on the need for the PRINT_MORE_FRAMES flag and
Pedro> having it based on from_tty?  I assume that your first made
Pedro> the printing unconditional, but then for some reason decided
Pedro> against it?

I think my reason was just to have it parallel the no-frame-filter code
in stack.c:

      /* If we've stopped before the end, mention that.  */
      if (fi && from_tty)
	printf_filtered (_("(More stack frames follow...)\n"));

I don't know why this code is conditional on from_tty, but that seemed
like a separate decision.

Tom
  

>>>>> "Tom" == Tom Tromey <tom@tromey.com> writes:

>> PR python/16497 notes that using "bt" with a positive argument prints
>> the wrong number of frames when a frame filter is in use.

Tom> Going through my un-reviewed patches today.
Tom> Ping for this one.

I don't think this was ever approved.

There's a follow-up as well (to change the enum to use
DEF_ENUM_FLAGS_TYPE), but that one was OK'd; but still pending since it
touched the same code as this patch.

Tom
  

>>>>> "Tom" == Tom Tromey <tom@tromey.com> writes:

>>>>> "Tom" == Tom Tromey <tom@tromey.com> writes:
>>> PR python/16497 notes that using "bt" with a positive argument prints
>>> the wrong number of frames when a frame filter is in use.

Tom> Going through my un-reviewed patches today.
Tom> Ping for this one.

Tom> I don't think this was ever approved.

Tom> There's a follow-up as well (to change the enum to use
Tom> DEF_ENUM_FLAGS_TYPE), but that one was OK'd; but still pending since it
Tom> touched the same code as this patch.

This is still pending review.

Some discussion here:
https://sourceware.org/ml/gdb-patches/2017-07/msg00200.html

Original patch here:
https://sourceware.org/ml/gdb-patches/2017-04/msg00630.html

I have a branch with this patch plus the previously-approved patch to
change frame_filter_flags to use DEF_ENUM_FLAGS_TYPE, so if/when this
one is ok'd, both will go in.

thanks,
Tom
  

>>>>> "Tom" == Tom Tromey <tom@tromey.com> writes:

Tom> There's a follow-up as well (to change the enum to use
Tom> DEF_ENUM_FLAGS_TYPE), but that one was OK'd; but still pending since it
Tom> touched the same code as this patch.

Tom> This is still pending review.

Tom> Some discussion here:
Tom> https://sourceware.org/ml/gdb-patches/2017-07/msg00200.html

Tom> Original patch here:
Tom> https://sourceware.org/ml/gdb-patches/2017-04/msg00630.html

Tom> I have a branch with this patch plus the previously-approved patch to
Tom> change frame_filter_flags to use DEF_ENUM_FLAGS_TYPE, so if/when this
Tom> one is ok'd, both will go in.

Ping again.

Tom
  

On 2018-02-24 12:09 PM, Tom Tromey wrote:
>>>>>> "Tom" == Tom Tromey <tom@tromey.com> writes:
> 
> Tom> There's a follow-up as well (to change the enum to use
> Tom> DEF_ENUM_FLAGS_TYPE), but that one was OK'd; but still pending since it
> Tom> touched the same code as this patch.
> 
> Tom> This is still pending review.
> 
> Tom> Some discussion here:
> Tom> https://sourceware.org/ml/gdb-patches/2017-07/msg00200.html
> 
> Tom> Original patch here:
> Tom> https://sourceware.org/ml/gdb-patches/2017-04/msg00630.html
> 
> Tom> I have a branch with this patch plus the previously-approved patch to
> Tom> change frame_filter_flags to use DEF_ENUM_FLAGS_TYPE, so if/when this
> Tom> one is ok'd, both will go in.
> 
> Ping again.
> 
> Tom
> 

The patch LGTM, considering all previously made comments are addressed.

Did you post the patch that makes frame_filter_frames an enum flags type?  I
don't see it.  If you think it's obvious enough to push it directly, make sure
to post it on the ML afterwards.

I also noticed that the frame filter flags are documented at many different places:

- above gdbpy_apply_frame_filter
- above extension_language_ops::apply_frame_filter
- above apply_ext_lang_frame_filter
- in enum frame_filter_flags

This is a recipe for them to become out of date, so I think we should only have
it in the enum declaration.  We can address that after your patches have merged.

Simon
  

>>>>> "Simon" == Simon Marchi <simark@simark.ca> writes:

Simon> The patch LGTM, considering all previously made comments are
Simon> addressed.

I believe they've all been but I will double check.

Simon> Did you post the patch that makes frame_filter_frames an enum
Simon> flags type?  I don't see it.

Yeah, it was here:

https://sourceware.org/ml/gdb-patches/2017-04/msg00760.html

A bit hard to find.

Tom