doc: Add table of MI versions

  This patch adds a table summarizing the history or MI versions:

- The version number
- Which GDB version introduced it
- Breaking changes compared to the previous version

The goal of the table is to help writers of front ends know which
version of MI they can use with a given GDB version.  It will also help
them update their code to work against a newer MI version.

Right now, we just have 1 and 2, but we expect to add an entry for 3
soon.  I did a bit of archelogy and reverse engineering of the code to
come up with the breaking changes for MI 2.

I did some changes to the text around it, some things that I thought
needed to be clarified, seemed a bit dated or seemed just wrong
(especially "Apart from mi0, new versions of @value{GDBN} will not
support old versions of MI").

gdb/doc/ChangeLog:

	* gdb.texinfo (GDB/MI Development and Front Ends): Add table of
	MI versions.  Update text around it.
---
 gdb/doc/gdb.texinfo | 57 ++++++++++++++++++++++++++++++++++++---------
 1 file changed, 46 insertions(+), 11 deletions(-)
  

> From: Simon Marchi <simon.marchi@ericsson.com>
> CC: Simon Marchi <simon.marchi@ericsson.com>
> Date: Mon, 14 Jan 2019 20:39:16 +0000
> 
> This patch adds a table summarizing the history or MI versions:
> 
> - The version number
> - Which GDB version introduced it
> - Breaking changes compared to the previous version

Thanks.

> -Although @sc{gdb/mi} is still incomplete, it is currently being used
> -by a variety of front ends to @value{GDBN}.  This makes it difficult
> -to introduce new functionality without breaking existing usage.  This
> -section tries to minimize the problems by describing how the protocol
> -might change.
> +The MI interface is versioned, allowing it to evolve while avoiding breaking
> +existing front ends.

Some of the rationale you removed sounds like something good to have.
Explaining the rationale for a section is in general a Good Thing,
IMO.

>  If the changes are likely to break front ends, the MI version level
> -will be increased by one.  This will allow the front end to parse the
> -output according to the MI version.  Apart from mi0, new versions of
> -@value{GDBN} will not support old versions of MI and it will be the
> -responsibility of the front end to work with the new one.
> +will be increased by one.  Previous versions of MI remain available, allowing
> +front ends to keep using them until they are modified to use the latest MI
> +version.

Likewise here: the old text explained that miN version will generally
be incompatible with miN-1 version.  Your change removes that
important statement.  I'd prefer not to lose that part.

> -@c Starting with mi3, add a new command -mi-version that prints the MI
> -@c version?

Why did you remove the comment?  It seems like a valid idea, perhaps
worth implementing.

> +Since @code{--interpreter=mi} always points to the latest MI version, it is
> +recommended that front ends request a specific version of MI when launching
> +@value{GDBN} (e.g. @code{--interpreter=mi2}) to make sure to get an interpreter
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
"to make sure they get and interpreter ..."

> +The @code{-environment-pwd}, @code{-environment-directory} and
> +@code{-environment-path} commands now returns values using the MI output
> +syntax, rather than CLI output.
                       ^^^^^^^^^^
"CLI output syntax", I presume.

> +@item
> +@code{-var-list-children}'s @code{children} result field is a now list, rather
                                                            ^^^^^^^^
A typo.
  

>> -Although @sc{gdb/mi} is still incomplete, it is currently being used
>> -by a variety of front ends to @value{GDBN}.  This makes it difficult
>> -to introduce new functionality without breaking existing usage.  This
>> -section tries to minimize the problems by describing how the protocol
>> -might change.
>> +The MI interface is versioned, allowing it to evolve while avoiding 
>> breaking
>> +existing front ends.
> 
> Some of the rationale you removed sounds like something good to have.
> Explaining the rationale for a section is in general a Good Thing,
> IMO.

The "is still incomplete" sentence sounds useless to me, and can even 
make people wonder if they should really use it, since it's incomplete.  
It will always evolve, it will never be "complete".  I could add back 
the last sentence with a bit more stuff, like so:

The MI interface is versioned, allowing it to evolve while avoiding 
breaking existing front ends.  This section describes how the protocol 
might change within a version and the breaking changes across versions.

>>  If the changes are likely to break front ends, the MI version level
>> -will be increased by one.  This will allow the front end to parse the
>> -output according to the MI version.  Apart from mi0, new versions of
>> -@value{GDBN} will not support old versions of MI and it will be the
>> -responsibility of the front end to work with the new one.
>> +will be increased by one.  Previous versions of MI remain available, 
>> allowing
>> +front ends to keep using them until they are modified to use the 
>> latest MI
>> +version.
> 
> Likewise here: the old text explained that miN version will generally
> be incompatible with miN-1 version.  Your change removes that
> important statement.  I'd prefer not to lose that part.

Which part of the original text says that?  The first sentence, which I 
have not removed, is pretty clear about that.

And the statement "Apart from mi0, new versions of GDB will not support 
old versions of MI" sounds completely wrong.  New versions of GDB don't 
support mi0 (presumably the first version of MI to have existed), but do 
support mi1.  What I want to say is that even if a new version of MI is 
released, the previous versions of MI stay available for some time, 
allowing front ends to do the transition.

>> -@c Starting with mi3, add a new command -mi-version that prints the 
>> MI
>> -@c version?
> 
> Why did you remove the comment?  It seems like a valid idea, perhaps
> worth implementing.

I don't think this is the right place for such things (it's quite 
hidden).  If we really want to keep track of this, let's open an issue 
on Bugzilla for it.

About the idea itself, I don't think we need to implement this.  If 
front ends request a specific version of MI (which is good practice, in 
my experience), they won't need to query it.

>> +Since @code{--interpreter=mi} always points to the latest MI version, 
>> it is
>> +recommended that front ends request a specific version of MI when 
>> launching
>> +@value{GDBN} (e.g. @code{--interpreter=mi2}) to make sure to get an 
>> interpreter
> 
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> "to make sure they get and interpreter ..."

Fixed.

>> +The @code{-environment-pwd}, @code{-environment-directory} and
>> +@code{-environment-path} commands now returns values using the MI 
>> output
>> +syntax, rather than CLI output.
>                        ^^^^^^^^^^
> "CLI output syntax", I presume.

Not sure there is really a syntax for CLI output like there is a syntax 
for MI output, but I made the change anyway.

>> +@item
>> +@code{-var-list-children}'s @code{children} result field is a now 
>> list, rather
>                                                             ^^^^^^^^
> A typo.

Fixed.

Thanks,

Simon
  

> Date: Tue, 15 Jan 2019 13:27:37 -0500
> From: Simon Marchi <simon.marchi@polymtl.ca>
> Cc: Simon Marchi <simon.marchi@ericsson.com>, gdb-patches@sourceware.org
> 
> >> -Although @sc{gdb/mi} is still incomplete, it is currently being used
> >> -by a variety of front ends to @value{GDBN}.  This makes it difficult
> >> -to introduce new functionality without breaking existing usage.  This
> >> -section tries to minimize the problems by describing how the protocol
> >> -might change.
> >> +The MI interface is versioned, allowing it to evolve while avoiding 
> >> breaking
> >> +existing front ends.
> > 
> > Some of the rationale you removed sounds like something good to have.
> > Explaining the rationale for a section is in general a Good Thing,
> > IMO.
> 
> The "is still incomplete" sentence sounds useless to me, and can even 
> make people wonder if they should really use it, since it's incomplete.  
> It will always evolve, it will never be "complete".  I could add back 
> the last sentence with a bit more stuff, like so:
> 
> The MI interface is versioned, allowing it to evolve while avoiding 
> breaking existing front ends.  This section describes how the protocol 
> might change within a version and the breaking changes across versions.

The part about MI being incomplete is not what I wanted to preserve.
How about something like this:

  Since @sc{gdb/mi} is used by a variety of front ends to
  @value{GDBN}, introduction of new MI functionality almost always
  breaks existing usage.  This section describes how the protocol
  changes and how to request previous version of the protocol when it
  does.

> >>  If the changes are likely to break front ends, the MI version level
> >> -will be increased by one.  This will allow the front end to parse the
> >> -output according to the MI version.  Apart from mi0, new versions of
> >> -@value{GDBN} will not support old versions of MI and it will be the
> >> -responsibility of the front end to work with the new one.
> >> +will be increased by one.  Previous versions of MI remain available, 
> >> allowing
> >> +front ends to keep using them until they are modified to use the 
> >> latest MI
> >> +version.
> > 
> > Likewise here: the old text explained that miN version will generally
> > be incompatible with miN-1 version.  Your change removes that
> > important statement.  I'd prefer not to lose that part.
> 
> Which part of the original text says that?

This one:

  [...] new versions of @value{GDBN} will not support old versions of
  MI.

Which is actually slightly confusing; a better way of saying that is
something like

  new versions of the MI protocol are not compatible with the old
  versions

> What I want to say is that even if a new version of MI is 
> released, the previous versions of MI stay available for some time, 
> allowing front ends to do the transition.

That's okay, but it's a separate issue.  Let's keep the other
information as well, as suggested above.

> >> -@c Starting with mi3, add a new command -mi-version that prints the 
> >> MI
> >> -@c version?
> > 
> > Why did you remove the comment?  It seems like a valid idea, perhaps
> > worth implementing.
> 
> I don't think this is the right place for such things (it's quite 
> hidden).  If we really want to keep track of this, let's open an issue 
> on Bugzilla for it.

I'm fine with moving this to bugzilla, I just don't want to lose the
suggestion altogether, as part of unrelated changes on top of that.

> About the idea itself, I don't think we need to implement this.

We don't need to agree with it, we just need to preserve the
suggestion.

> If front ends request a specific version of MI (which is good
> practice, in my experience), they won't need to query it.

What if a front end can support several versions, provided that it
knows the latest version which is provided?  Why require such a front
end to request the lowest common denominator, instead of adapting to
the latest version it can support?

Thanks.
  

On 2019-01-15 14:19, Eli Zaretskii wrote:
>> Date: Tue, 15 Jan 2019 13:27:37 -0500
>> From: Simon Marchi <simon.marchi@polymtl.ca>
>> Cc: Simon Marchi <simon.marchi@ericsson.com>, 
>> gdb-patches@sourceware.org
>> 
>> >> -Although @sc{gdb/mi} is still incomplete, it is currently being used
>> >> -by a variety of front ends to @value{GDBN}.  This makes it difficult
>> >> -to introduce new functionality without breaking existing usage.  This
>> >> -section tries to minimize the problems by describing how the protocol
>> >> -might change.
>> >> +The MI interface is versioned, allowing it to evolve while avoiding
>> >> breaking
>> >> +existing front ends.
>> >
>> > Some of the rationale you removed sounds like something good to have.
>> > Explaining the rationale for a section is in general a Good Thing,
>> > IMO.
>> 
>> The "is still incomplete" sentence sounds useless to me, and can even
>> make people wonder if they should really use it, since it's 
>> incomplete.
>> It will always evolve, it will never be "complete".  I could add back
>> the last sentence with a bit more stuff, like so:
>> 
>> The MI interface is versioned, allowing it to evolve while avoiding
>> breaking existing front ends.  This section describes how the protocol
>> might change within a version and the breaking changes across 
>> versions.
> 
> The part about MI being incomplete is not what I wanted to preserve.
> How about something like this:
> 
>   Since @sc{gdb/mi} is used by a variety of front ends to
>   @value{GDBN}, introduction of new MI functionality almost always
>   breaks existing usage.  This section describes how the protocol
>   changes and how to request previous version of the protocol when it
>   does.

I do not think that is accurate.  Most of the time, we introduce new MI 
functionality without breaking existing usage.  We can do so precisely 
because of what it described in this section.  In particular, that a 
front end should handle gracefully commands and fields it doesn't know 
about.  It's more when we realized we messed up and we must remove or 
change something that is already there, that we break existing usage.

Therefore, I would suggest a little change to your paragraph:

Since @sc{gdb/mi} is used by a variety of front ends to
@value{GDBN}, changes to the MI interface may break existing usage.
This section describes how the protocol changes and how to request
previous version of the protocol when it does.

>> >>  If the changes are likely to break front ends, the MI version level
>> >> -will be increased by one.  This will allow the front end to parse the
>> >> -output according to the MI version.  Apart from mi0, new versions of
>> >> -@value{GDBN} will not support old versions of MI and it will be the
>> >> -responsibility of the front end to work with the new one.
>> >> +will be increased by one.  Previous versions of MI remain available,
>> >> allowing
>> >> +front ends to keep using them until they are modified to use the
>> >> latest MI
>> >> +version.
>> >
>> > Likewise here: the old text explained that miN version will generally
>> > be incompatible with miN-1 version.  Your change removes that
>> > important statement.  I'd prefer not to lose that part.
>> 
>> Which part of the original text says that?
> 
> This one:
> 
>   [...] new versions of @value{GDBN} will not support old versions of
>   MI.
> 
> Which is actually slightly confusing; a better way of saying that is
> something like
> 
>   new versions of the MI protocol are not compatible with the old
>   versions

I thought this was quite obvious by the fact that we say that we 
introduce a new version when we make breaking changes.  But I can add 
this sentence, which would result in this:

If the changes are likely to break front ends, the MI version level
will be increased by one.  The new versions of the MI protocol are not 
compatible
with the old versions.  Old versions of MI remain available, allowing 
front ends
to keep using them until they are modified to use the latest MI version.

>> >> -@c Starting with mi3, add a new command -mi-version that prints the
>> >> MI
>> >> -@c version?
>> >
>> > Why did you remove the comment?  It seems like a valid idea, perhaps
>> > worth implementing.
>> 
>> I don't think this is the right place for such things (it's quite
>> hidden).  If we really want to keep track of this, let's open an issue
>> on Bugzilla for it.
> 
> I'm fine with moving this to bugzilla, I just don't want to lose the
> suggestion altogether, as part of unrelated changes on top of that.
> 
>> About the idea itself, I don't think we need to implement this.
> 
> We don't need to agree with it, we just need to preserve the
> suggestion.

I have opened [1], is it fine to remove the comment from gdb.texinfo?

>> If front ends request a specific version of MI (which is good
>> practice, in my experience), they won't need to query it.
> 
> What if a front end can support several versions, provided that it
> knows the latest version which is provided?  Why require such a front
> end to request the lowest common denominator, instead of adapting to
> the latest version it can support?

I don't think we require front ends to use the lowest common 
denominator.  Instead, it should request 
max(version_known_by_the_front_end, version_known_by_gdb).

Simon
  

> Date: Tue, 15 Jan 2019 15:37:45 -0500
> From: Simon Marchi <simon.marchi@polymtl.ca>
> Cc: simon.marchi@ericsson.com, gdb-patches@sourceware.org
> 
> Since @sc{gdb/mi} is used by a variety of front ends to
> @value{GDBN}, changes to the MI interface may break existing usage.
> This section describes how the protocol changes and how to request
> previous version of the protocol when it does.

OK, thanks.

> >   new versions of the MI protocol are not compatible with the old
> >   versions
> 
> I thought this was quite obvious by the fact that we say that we 
> introduce a new version when we make breaking changes.  But I can add 
> this sentence, which would result in this:
> 
> If the changes are likely to break front ends, the MI version level
> will be increased by one.  The new versions of the MI protocol are not 
> compatible
> with the old versions.  Old versions of MI remain available, allowing 
> front ends
> to keep using them until they are modified to use the latest MI version.

Fine by me.

> >> About the idea itself, I don't think we need to implement this.
> > 
> > We don't need to agree with it, we just need to preserve the
> > suggestion.
> 
> I have opened [1], is it fine to remove the comment from gdb.texinfo?

No objections from me.

> >> If front ends request a specific version of MI (which is good
> >> practice, in my experience), they won't need to query it.
> > 
> > What if a front end can support several versions, provided that it
> > knows the latest version which is provided?  Why require such a front
> > end to request the lowest common denominator, instead of adapting to
> > the latest version it can support?
> 
> I don't think we require front ends to use the lowest common 
> denominator.  Instead, it should request 
> max(version_known_by_the_front_end, version_known_by_gdb).

And I think version_known_by_gdb needs this command, doesn't it?

Thanks.
  

On 2019-01-16 12:03, Eli Zaretskii wrote:
>> I don't think we require front ends to use the lowest common
>> denominator.  Instead, it should request
>> max(version_known_by_the_front_end, version_known_by_gdb).
> 
> And I think version_known_by_gdb needs this command, doesn't it?

There are multiple strategies to find this information, this could be 
one.  A front end could start GDB with "-i=mi" and issue -mi-version to 
get it.  It can then determine whether that version is satisfactory or 
if it should restart GDB with an earlier version of MI.

Another strategy, for a frontend that knows mi1, mi2 and mi3, could be 
to try to start GDB with "-i=mi3" first.  If GDB exits immediately with 
"Interpreter `mi3' unrecognized", try with mi2, and so forth.

The frontends I have worked with parse the output of "gdb --version" to 
find the GDB version, so they could deduce which MI version to use based 
on that.  It's not pretty, since different builds of GDB can have 
different looking version strings but it works well enough.  Knowing the 
GDB version is usually needed anyway, because they need to work around a 
variety of bugs, not necessarily related to MI.

Simon
  

On Wed, Jan 16, 2019 at 12:21:17PM -0500, Simon Marchi wrote:
> On 2019-01-16 12:03, Eli Zaretskii wrote:
> > > I don't think we require front ends to use the lowest common
> > > denominator.  Instead, it should request
> > > max(version_known_by_the_front_end, version_known_by_gdb).
> > 
> > And I think version_known_by_gdb needs this command, doesn't it?
> 
> There are multiple strategies to find this information, this could be one.
> A front end could start GDB with "-i=mi" and issue -mi-version to get it.
> It can then determine whether that version is satisfactory or if it should
> restart GDB with an earlier version of MI.
> 
> Another strategy, for a frontend that knows mi1, mi2 and mi3, could be to
> try to start GDB with "-i=mi3" first.  If GDB exits immediately with
> "Interpreter `mi3' unrecognized", try with mi2, and so forth.

Sounds unlikely to me. I'd rather expect frontends to simply cope with
all versions of possible outputs they are aware of. The case that started
this discussion here is straight-forward to handle, for instance.

> The frontends I have worked with parse the output of "gdb --version" to find
> the GDB version,  so they could deduce which MI version to use based on that.
> It's not pretty, since different builds of GDB can have different looking
> version strings but it works well enough.

That's impressive. I've seen e.g. version strings like

    GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
    GNU gdb (GDB) 7.0.1-debian
    GNU gdb (GDB) 7.0.90.20100226-cvs
    GNU gdb (GDB) 7.1-ubuntu
    GNU gdb (GDB) 7.3 qnx (rev. 613)
    GNU gdb (GDB) Fedora (7.1-22.fc13)
    GNU gdb (GDB) Fedora 8.0-13.fc26
    GNU gdb (GDB) SUSE (6.8.91.20090930-2.4)
    GNU gdb (GDB; SUSE Linux Enterprise 10) 7.9.1
    GNU gdb (GDB; openSUSE 13.1) 7.6.50.20130731-cvs
    GNU gdb (Gentoo 7.1 p1) 7.1
    GNU gdb (rubenvb-4.7.2-release) 7.5.50.20120920-cvs
    GNU gdb 6.3.50-20050815 (Apple version gdb-1461.2)
    GNU gdb Fedora (6.8-37.el5)

Yes, I can try to parse a GDB release from that, and I can guess
on available features from that but I never ended up with something
reasonable robust that way.

One cannot even assume that the same base GDB release works the same 
everywhere as different distributions pile different local patches on top.

> Knowing the GDB version is usually needed anyway, because they
> need to work around a variety of bugs, not necessarily related to MI.

"Feature discovery" can also work by trial-and-error: Try one syntax,
if that fails, take the next one. This can even be cached per session
in most cases.

Andre'