manual: scripts/documented.sh updated

  Hello,
As per the Missing Functions section of master Todo list[1],
scripts/documented.sh needs to be updated.

Could someone please review this patch?
It removes the reference to linuxthreads/*.taxi files from the script
and removes the individual email address from the output.

Thanks!
Girish Joshi
girishjoshi.io

[1]: https://sourceware.org/glibc/wiki/Development_Todo/Master
  

* Girish Joshi via Libc-alpha:

> Hello,
> As per the Missing Functions section of master Todo list[1],
> scripts/documented.sh needs to be updated.
>
> Could someone please review this patch?
> It removes the reference to linuxthreads/*.taxi files from the script
> and removes the individual email address from the output.

The script still doesn't seem to work, probably because the symbol names
are not extract properly:

43120 T _dl_mcount_wrapper 	43140 T _dl_mcount_wrapper_c 	43320 W clogf 	43320 W clogf32
435c0 T _dl_find_object 	43740 T realpath 	43770 T quick_exit 	43790 T _IO_vfscanf
437c0 T _pthread_cleanup_pus 	437d0 T _pthread_cleanup_pop 	437d0 W cprojf 	437d0 W cprojf32

_dl_find_object is documented.

_IO_vfscanf should be recognized as an alias of vfscanf, but this is
more of an RFE than a bug fix.

Thanks,
Florian
  

Hi Florian,
Thanks for the review.
Turns out `diff` was not working with the given input files, this
patch fixes that.
I'm not sure if there is a list of people who volunteered to document
some specific functions.
So that comment from the output html is removed.

Thanks!
Girish Joshi

On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Girish Joshi via Libc-alpha:
>
> > Hello,
> > As per the Missing Functions section of master Todo list[1],
> > scripts/documented.sh needs to be updated.
> >
> > Could someone please review this patch?
> > It removes the reference to linuxthreads/*.taxi files from the script
> > and removes the individual email address from the output.
>
> The script still doesn't seem to work, probably because the symbol names
> are not extract properly:
>
> 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
> 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
> 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
>
> _dl_find_object is documented.
>
> _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> more of an RFE than a bug fix.
>
> Thanks,
> Florian
>
  

Also,
> On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
> _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> more of an RFE than a bug fix.
I'll try to figure out a way to mark such functions as aliases, as of
now the script does not do that.

Thanks,
Girish Joshi


On Sun, Mar 5, 2023 at 11:44 AM Girish Joshi <girish946@gmail.com> wrote:
>
> Hi Florian,
> Thanks for the review.
> Turns out `diff` was not working with the given input files, this
> patch fixes that.
> I'm not sure if there is a list of people who volunteered to document
> some specific functions.
> So that comment from the output html is removed.
>
> Thanks!
> Girish Joshi
>

> >
> > * Girish Joshi via Libc-alpha:
> >
> > > Hello,
> > > As per the Missing Functions section of master Todo list[1],
> > > scripts/documented.sh needs to be updated.
> > >
> > > Could someone please review this patch?
> > > It removes the reference to linuxthreads/*.taxi files from the script
> > > and removes the individual email address from the output.
> >
> > The script still doesn't seem to work, probably because the symbol names
> > are not extract properly:
> >
> > 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
> > 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
> > 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
> >
> > _dl_find_object is documented.
> >
> > _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> > more of an RFE than a bug fix.
> >
> > Thanks,
> > Florian
> >
  

On Sun, Mar 05, 2023 at 11:44:22AM +0530, Girish Joshi via Libc-alpha wrote:
> Hi Florian,
> Thanks for the review.
> Turns out `diff` was not working with the given input files, this
> patch fixes that.
> I'm not sure if there is a list of people who volunteered to document
> some specific functions.
> So that comment from the output html is removed.
> 
> Thanks!
> Girish Joshi
> 
> On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
> >
> > * Girish Joshi via Libc-alpha:
> >
> > > Hello,
> > > As per the Missing Functions section of master Todo list[1],
> > > scripts/documented.sh needs to be updated.
> > >
> > > Could someone please review this patch?
> > > It removes the reference to linuxthreads/*.taxi files from the script
> > > and removes the individual email address from the output.
> >
> > The script still doesn't seem to work, probably because the symbol names
> > are not extract properly:
> >
> > 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
> > 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
> > 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
> >
> > _dl_find_object is documented.
> >
> > _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> > more of an RFE than a bug fix.
> >
> > Thanks,
> > Florian
> >

> From 5148c65ac07688dc5c3ee79a9b0f054232d64df3 Mon Sep 17 00:00:00 2001
> From: Girish Joshi <girish946@gmail.com>
> Date: Sun, 5 Mar 2023 11:15:00 +0530
> Subject: [PATCH] manual: scripts/documented.sh updated
> 
> References for linuxthreads are removed from scripts/documented.sh.
> Also removed the individual email address from the html output.
> 
> Fixed the steps to retrive undocumented functions
> ---
>  scripts/documented.sh | 44 +++++++++++++++++++++++++++++++------------
>  1 file changed, 32 insertions(+), 12 deletions(-)
>  mode change 100644 => 100755 scripts/documented.sh
> 
> diff --git a/scripts/documented.sh b/scripts/documented.sh
> old mode 100644
> new mode 100755
> index 7d1ffd4a92..dda6a32283
> --- a/scripts/documented.sh
> +++ b/scripts/documented.sh
> @@ -1,9 +1,15 @@
>  #!/bin/sh
>  bindir=$1
>  
> -VERSION=1.0
> +# usage: scripts/documented.sh <path to glibc build directory>
> +# for example:
> +# scripts/documented.sh $HOME/build/glibc
>  
> -grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
> +VERSION=1.1
> +
> +# Collect the documented functions from manual/*.texi files.
> +
> +grep -E -h @deftypefu?nx? manual/*.texi |
>  sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
>  sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
>  sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
> @@ -12,9 +18,12 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
>  sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
>  sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
>  sed -e '/^alloca$/d' |
> +sed -r '/^\s*$/d' |
>  sort -u > DOCUMENTED
>  
> -nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
> +
> +# extract the exported symbols from the libraries.
> +nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
>  grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
>  sed 's/\(@.*\)//' |
>  cut -b 12- |

You can change this to 'cut -v 20-' and avoid the extra
AVAILABLE_WITHOUT_OFFSET part.

Thanks,
Joe
> @@ -25,6 +34,9 @@ sed -e '/^_mcleanup$/d' -e '/^_rpc_dtablesize$/d' -e '/^_seterr_reply$/d' |
>  sed -e '/^_nss/d' -e '/^_setjmp$/d' |
>  sort -u > AVAILABLE
>  
> +# remove offset details from nm output
> +cat AVAILABLE | cut -b 9-  | sort -u  > AVAILABLE_WITHOUT_OFFSET
> +
>  cat <<EOF
>  <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
>  <html>
> @@ -39,9 +51,7 @@ cat <<EOF
>      which are not yet documented in the manual.  This list is
>      automatically created and therefore might contain errors.  Please
>      check the latest manual (available from the CVS archive) before
> -    starting to work.  It might also be good to let me know in
> -    advanace on which functions you intend to work to avoid
> -    duplication.</p>
> +    starting to work.</p>
>  
>      <p>A few comments:</p>
>  
> @@ -52,15 +62,23 @@ cat <<EOF
>        <li>Similarly for the LFS functions (those ending in 64).</li>
>      </ul>
>  
> -    <p>The function sombody already volunteered to document are marked
> -    with a reference to the person.</p>
>  
>      <center><table>
>  EOF
>  
>  n=0
> -diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
> -expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
> +
> +# get the list of undocumented functions
> +grep -Fvx -f DOCUMENTED AVAILABLE_WITHOUT_OFFSET > UNDOCUMENTED_WITHOUT_OFFSET
> +
> +# get undocumented functions with offset info
> +for func in `cat UNDOCUMENTED_WITHOUT_OFFSET`
> +do
> +  grep " $func$" AVAILABLE >> UNDOCUMENTED
> +done
> +
> +diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
> +expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
>  while read name; do
>    line="$line
>  <td><tt>$name</tt></td>"
> @@ -95,8 +113,10 @@ cat <<EOF
>      </table></center>
>  
>      <hr>
> -    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
> -Generated on $(date) with documented.sh version $VERSION
> +    Generated on $(date) with documented.sh version $VERSION
>    </body>
>  </html>
>  EOF
> +
> +# cleanup the files
> +rm -rf UNDOCUMENTED_WITHOUT_OFFSET AVAILABLE_WITHOUT_OFFSET AVAILABLE
> -- 
> 2.39.2
>
  

Thanks Joe, I've updated the script.
It's just that the output now does not contain the addresses and the
T/W info about the symbol is not present.
Do we need that info about each symbol in the output?

Thanks!
Girish Joshi
girishjoshi.io

On Sat, Mar 11, 2023 at 8:18 PM Joe Simmons-Talbott
<joetalbott@gmail.com> wrote:
>
> On Sun, Mar 05, 2023 at 11:44:22AM +0530, Girish Joshi via Libc-alpha wrote:
> > Hi Florian,
> > Thanks for the review.
> > Turns out `diff` was not working with the given input files, this
> > patch fixes that.
> > I'm not sure if there is a list of people who volunteered to document
> > some specific functions.
> > So that comment from the output html is removed.
> >
> > Thanks!
> > Girish Joshi
> >
> > On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
> > >
> > > * Girish Joshi via Libc-alpha:
> > >
> > > > Hello,
> > > > As per the Missing Functions section of master Todo list[1],
> > > > scripts/documented.sh needs to be updated.
> > > >
> > > > Could someone please review this patch?
> > > > It removes the reference to linuxthreads/*.taxi files from the script
> > > > and removes the individual email address from the output.
> > >
> > > The script still doesn't seem to work, probably because the symbol names
> > > are not extract properly:
> > >
> > > 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
> > > 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
> > > 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
> > >
> > > _dl_find_object is documented.
> > >
> > > _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> > > more of an RFE than a bug fix.
> > >
> > > Thanks,
> > > Florian
> > >
>
> > From 5148c65ac07688dc5c3ee79a9b0f054232d64df3 Mon Sep 17 00:00:00 2001
> > From: Girish Joshi <girish946@gmail.com>
> > Date: Sun, 5 Mar 2023 11:15:00 +0530
> > Subject: [PATCH] manual: scripts/documented.sh updated
> >
> > References for linuxthreads are removed from scripts/documented.sh.
> > Also removed the individual email address from the html output.
> >
> > Fixed the steps to retrive undocumented functions
> > ---
> >  scripts/documented.sh | 44 +++++++++++++++++++++++++++++++------------
> >  1 file changed, 32 insertions(+), 12 deletions(-)
> >  mode change 100644 => 100755 scripts/documented.sh
> >
> > diff --git a/scripts/documented.sh b/scripts/documented.sh
> > old mode 100644
> > new mode 100755
> > index 7d1ffd4a92..dda6a32283
> > --- a/scripts/documented.sh
> > +++ b/scripts/documented.sh
> > @@ -1,9 +1,15 @@
> >  #!/bin/sh
> >  bindir=$1
> >
> > -VERSION=1.0
> > +# usage: scripts/documented.sh <path to glibc build directory>
> > +# for example:
> > +# scripts/documented.sh $HOME/build/glibc
> >
> > -grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
> > +VERSION=1.1
> > +
> > +# Collect the documented functions from manual/*.texi files.
> > +
> > +grep -E -h @deftypefu?nx? manual/*.texi |
> >  sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
> >  sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
> >  sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
> > @@ -12,9 +18,12 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
> >  sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
> >  sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
> >  sed -e '/^alloca$/d' |
> > +sed -r '/^\s*$/d' |
> >  sort -u > DOCUMENTED
> >
> > -nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
> > +
> > +# extract the exported symbols from the libraries.
> > +nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
> >  grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
> >  sed 's/\(@.*\)//' |
> >  cut -b 12- |
>
> You can change this to 'cut -v 20-' and avoid the extra
> AVAILABLE_WITHOUT_OFFSET part.
>
> Thanks,
> Joe
> > @@ -25,6 +34,9 @@ sed -e '/^_mcleanup$/d' -e '/^_rpc_dtablesize$/d' -e '/^_seterr_reply$/d' |
> >  sed -e '/^_nss/d' -e '/^_setjmp$/d' |
> >  sort -u > AVAILABLE
> >
> > +# remove offset details from nm output
> > +cat AVAILABLE | cut -b 9-  | sort -u  > AVAILABLE_WITHOUT_OFFSET
> > +
> >  cat <<EOF
> >  <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
> >  <html>
> > @@ -39,9 +51,7 @@ cat <<EOF
> >      which are not yet documented in the manual.  This list is
> >      automatically created and therefore might contain errors.  Please
> >      check the latest manual (available from the CVS archive) before
> > -    starting to work.  It might also be good to let me know in
> > -    advanace on which functions you intend to work to avoid
> > -    duplication.</p>
> > +    starting to work.</p>
> >
> >      <p>A few comments:</p>
> >
> > @@ -52,15 +62,23 @@ cat <<EOF
> >        <li>Similarly for the LFS functions (those ending in 64).</li>
> >      </ul>
> >
> > -    <p>The function sombody already volunteered to document are marked
> > -    with a reference to the person.</p>
> >
> >      <center><table>
> >  EOF
> >
> >  n=0
> > -diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
> > -expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
> > +
> > +# get the list of undocumented functions
> > +grep -Fvx -f DOCUMENTED AVAILABLE_WITHOUT_OFFSET > UNDOCUMENTED_WITHOUT_OFFSET
> > +
> > +# get undocumented functions with offset info
> > +for func in `cat UNDOCUMENTED_WITHOUT_OFFSET`
> > +do
> > +  grep " $func$" AVAILABLE >> UNDOCUMENTED
> > +done
> > +
> > +diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
> > +expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
> >  while read name; do
> >    line="$line
> >  <td><tt>$name</tt></td>"
> > @@ -95,8 +113,10 @@ cat <<EOF
> >      </table></center>
> >
> >      <hr>
> > -    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
> > -Generated on $(date) with documented.sh version $VERSION
> > +    Generated on $(date) with documented.sh version $VERSION
> >    </body>
> >  </html>
> >  EOF
> > +
> > +# cleanup the files
> > +rm -rf UNDOCUMENTED_WITHOUT_OFFSET AVAILABLE_WITHOUT_OFFSET AVAILABLE
> > --
> > 2.39.2
> >
>
  

On Sat, Mar 30, 2024 at 2:54 PM Girish Joshi <girish946@gmail.com> wrote:
>
> Thanks Joe, I've updated the script.
> It's just that the output now does not contain the addresses and the
> T/W info about the symbol is not present.
> Do we need that info about each symbol in the output?

I guess that information is useful.  I'm attaching a patch that
reverts to 'cut -b 12' and does the grep and diff with a copy of
AVAILABLE that does not have the offsets and waits until it builds the
HTML table to get the matching line with the offset.

Thanks,
Joe
>
> Thanks!
> Girish Joshi
> girishjoshi.io
>
> On Sat, Mar 11, 2023 at 8:18 PM Joe Simmons-Talbott
> <joetalbott@gmail.com> wrote:
> >
> > On Sun, Mar 05, 2023 at 11:44:22AM +0530, Girish Joshi via Libc-alpha wrote:
> > > Hi Florian,
> > > Thanks for the review.
> > > Turns out `diff` was not working with the given input files, this
> > > patch fixes that.
> > > I'm not sure if there is a list of people who volunteered to document
> > > some specific functions.
> > > So that comment from the output html is removed.
> > >
> > > Thanks!
> > > Girish Joshi
> > >
> > > On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
> > > >
> > > > * Girish Joshi via Libc-alpha:
> > > >
> > > > > Hello,
> > > > > As per the Missing Functions section of master Todo list[1],
> > > > > scripts/documented.sh needs to be updated.
> > > > >
> > > > > Could someone please review this patch?
> > > > > It removes the reference to linuxthreads/*.taxi files from the script
> > > > > and removes the individual email address from the output.
> > > >
> > > > The script still doesn't seem to work, probably because the symbol names
> > > > are not extract properly:
> > > >
> > > > 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
> > > > 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
> > > > 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
> > > >
> > > > _dl_find_object is documented.
> > > >
> > > > _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> > > > more of an RFE than a bug fix.
> > > >
> > > > Thanks,
> > > > Florian
> > > >
> >
> > > From 5148c65ac07688dc5c3ee79a9b0f054232d64df3 Mon Sep 17 00:00:00 2001
> > > From: Girish Joshi <girish946@gmail.com>
> > > Date: Sun, 5 Mar 2023 11:15:00 +0530
> > > Subject: [PATCH] manual: scripts/documented.sh updated
> > >
> > > References for linuxthreads are removed from scripts/documented.sh.
> > > Also removed the individual email address from the html output.
> > >
> > > Fixed the steps to retrive undocumented functions
> > > ---
> > >  scripts/documented.sh | 44 +++++++++++++++++++++++++++++++------------
> > >  1 file changed, 32 insertions(+), 12 deletions(-)
> > >  mode change 100644 => 100755 scripts/documented.sh
> > >
> > > diff --git a/scripts/documented.sh b/scripts/documented.sh
> > > old mode 100644
> > > new mode 100755
> > > index 7d1ffd4a92..dda6a32283
> > > --- a/scripts/documented.sh
> > > +++ b/scripts/documented.sh
> > > @@ -1,9 +1,15 @@
> > >  #!/bin/sh
> > >  bindir=$1
> > >
> > > -VERSION=1.0
> > > +# usage: scripts/documented.sh <path to glibc build directory>
> > > +# for example:
> > > +# scripts/documented.sh $HOME/build/glibc
> > >
> > > -grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
> > > +VERSION=1.1
> > > +
> > > +# Collect the documented functions from manual/*.texi files.
> > > +
> > > +grep -E -h @deftypefu?nx? manual/*.texi |
> > >  sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
> > >  sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
> > >  sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
> > > @@ -12,9 +18,12 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
> > >  sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
> > >  sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
> > >  sed -e '/^alloca$/d' |
> > > +sed -r '/^\s*$/d' |
> > >  sort -u > DOCUMENTED
> > >
> > > -nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
> > > +
> > > +# extract the exported symbols from the libraries.
> > > +nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
> > >  grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
> > >  sed 's/\(@.*\)//' |
> > >  cut -b 12- |
> >
> > You can change this to 'cut -v 20-' and avoid the extra
> > AVAILABLE_WITHOUT_OFFSET part.
> >
> > Thanks,
> > Joe
> > > @@ -25,6 +34,9 @@ sed -e '/^_mcleanup$/d' -e '/^_rpc_dtablesize$/d' -e '/^_seterr_reply$/d' |
> > >  sed -e '/^_nss/d' -e '/^_setjmp$/d' |
> > >  sort -u > AVAILABLE
> > >
> > > +# remove offset details from nm output
> > > +cat AVAILABLE | cut -b 9-  | sort -u  > AVAILABLE_WITHOUT_OFFSET
> > > +
> > >  cat <<EOF
> > >  <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
> > >  <html>
> > > @@ -39,9 +51,7 @@ cat <<EOF
> > >      which are not yet documented in the manual.  This list is
> > >      automatically created and therefore might contain errors.  Please
> > >      check the latest manual (available from the CVS archive) before
> > > -    starting to work.  It might also be good to let me know in
> > > -    advanace on which functions you intend to work to avoid
> > > -    duplication.</p>
> > > +    starting to work.</p>
> > >
> > >      <p>A few comments:</p>
> > >
> > > @@ -52,15 +62,23 @@ cat <<EOF
> > >        <li>Similarly for the LFS functions (those ending in 64).</li>
> > >      </ul>
> > >
> > > -    <p>The function sombody already volunteered to document are marked
> > > -    with a reference to the person.</p>
> > >
> > >      <center><table>
> > >  EOF
> > >
> > >  n=0
> > > -diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
> > > -expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
> > > +
> > > +# get the list of undocumented functions
> > > +grep -Fvx -f DOCUMENTED AVAILABLE_WITHOUT_OFFSET > UNDOCUMENTED_WITHOUT_OFFSET
> > > +
> > > +# get undocumented functions with offset info
> > > +for func in `cat UNDOCUMENTED_WITHOUT_OFFSET`
> > > +do
> > > +  grep " $func$" AVAILABLE >> UNDOCUMENTED
> > > +done
> > > +
> > > +diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
> > > +expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
> > >  while read name; do
> > >    line="$line
> > >  <td><tt>$name</tt></td>"
> > > @@ -95,8 +113,10 @@ cat <<EOF
> > >      </table></center>
> > >
> > >      <hr>
> > > -    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
> > > -Generated on $(date) with documented.sh version $VERSION
> > > +    Generated on $(date) with documented.sh version $VERSION
> > >    </body>
> > >  </html>
> > >  EOF
> > > +
> > > +# cleanup the files
> > > +rm -rf UNDOCUMENTED_WITHOUT_OFFSET AVAILABLE_WITHOUT_OFFSET AVAILABLE
> > > --
> > > 2.39.2
> > >
> >
  

On Mon, Apr 1, 2024 at 10:03 PM Joe Talbott <joetalbott@gmail.com> wrote:
>
> I guess that information is useful.  I'm attaching a patch that
> reverts to 'cut -b 12' and does the grep and diff with a copy of
> AVAILABLE that does not have the offsets and waits until it builds the
> HTML table to get the matching line with the offset.

Thanks Joe,
I Just realized that this patch is archived on the patchwork. So the
latest changes are not reflected there.

Girish Joshi
girishjoshi.io
  

Hello,
Could someone please review this patch?

Thanks!
Girish Joshi
girishjoshi.io

On Tue, Apr 2, 2024 at 9:31 AM Girish Joshi <girish946@gmail.com> wrote:
>
> On Mon, Apr 1, 2024 at 10:03 PM Joe Talbott <joetalbott@gmail.com> wrote:
> >
> > I guess that information is useful.  I'm attaching a patch that
> > reverts to 'cut -b 12' and does the grep and diff with a copy of
> > AVAILABLE that does not have the offsets and waits until it builds the
> > HTML table to get the matching line with the offset.
>
> Thanks Joe,
> I Just realized that this patch is archived on the patchwork. So the
> latest changes are not reflected there.
>
> Girish Joshi
> girishjoshi.io
  

On 03/04/24 12:44, Girish Joshi wrote:
> Hello,
> Could someone please review this patch?
> 
> Thanks!
> Girish Joshi
> girishjoshi.io
> 
> On Tue, Apr 2, 2024 at 9:31 AM Girish Joshi <girish946@gmail.com> wrote:
>>
>> On Mon, Apr 1, 2024 at 10:03 PM Joe Talbott <joetalbott@gmail.com> wrote:
>>>
>>> I guess that information is useful.  I'm attaching a patch that
>>> reverts to 'cut -b 12' and does the grep and diff with a copy of
>>> AVAILABLE that does not have the offsets and waits until it builds the
>>> HTML table to get the matching line with the offset.
>>
>> Thanks Joe,
>> I Just realized that this patch is archived on the patchwork. So the
>> latest changes are not reflected there.
>>
>> Girish Joshi
>> girishjoshi.io

I will take a look.
  

On Wed, Apr 3, 2024 at 10:41 PM Adhemerval Zanella Netto
<adhemerval.zanella@linaro.org> wrote:
>
> I will take a look.
Thanks Adhemerval.

Girish Joshi
girishjoshi.io
  

Hello Florian,
On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
> _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> more of an RFE than a bug fix.

I was able to write a small python script that gets the list of
aliases in the given binaries and the html.
Not sure if this can be a separate script or should it be a part of
the same thing?
Please let me know if it can be a separate script, I'll post a patch for it.

Thanks!
Girish Joshi
girishjoshi.io

On Thu, Apr 4, 2024 at 7:28 PM Girish Joshi <girish946@gmail.com> wrote:
>
> On Wed, Apr 3, 2024 at 10:41 PM Adhemerval Zanella Netto
> <adhemerval.zanella@linaro.org> wrote:
> >
> > I will take a look.
> Thanks Adhemerval.
>
> Girish Joshi
> girishjoshi.io
  

ping.

Girish Joshi
girishjoshi.io

On Wed, Apr 3, 2024 at 10:41 PM Adhemerval Zanella Netto
<adhemerval.zanella@linaro.org> wrote:
>
>
>
> On 03/04/24 12:44, Girish Joshi wrote:
> > Hello,
> > Could someone please review this patch?
> >
> > Thanks!
> > Girish Joshi
> > girishjoshi.io
> >
> > On Tue, Apr 2, 2024 at 9:31 AM Girish Joshi <girish946@gmail.com> wrote:
> >>
> >> On Mon, Apr 1, 2024 at 10:03 PM Joe Talbott <joetalbott@gmail.com> wrote:
> >>>
> >>> I guess that information is useful.  I'm attaching a patch that
> >>> reverts to 'cut -b 12' and does the grep and diff with a copy of
> >>> AVAILABLE that does not have the offsets and waits until it builds the
> >>> HTML table to get the matching line with the offset.
> >>
> >> Thanks Joe,
> >> I Just realized that this patch is archived on the patchwork. So the
> >> latest changes are not reflected there.
> >>
> >> Girish Joshi
> >> girishjoshi.io
>
> I will take a look.
  

On 01/04/24 13:33, Joe Talbott wrote:
> On Sat, Mar 30, 2024 at 2:54 PM Girish Joshi <girish946@gmail.com> wrote:
>>
>> Thanks Joe, I've updated the script.
>> It's just that the output now does not contain the addresses and the
>> T/W info about the symbol is not present.
>> Do we need that info about each symbol in the output?
> 
> I guess that information is useful.  I'm attaching a patch that
> reverts to 'cut -b 12' and does the grep and diff with a copy of
> AVAILABLE that does not have the offsets and waits until it builds the
> HTML table to get the matching line with the offset.
> 
> Thanks,
> Joe
>>
>> Thanks!
>> Girish Joshi
>> girishjoshi.io
>>
>> On Sat, Mar 11, 2023 at 8:18 PM Joe Simmons-Talbott
>> <joetalbott@gmail.com> wrote:
>>>
>>> On Sun, Mar 05, 2023 at 11:44:22AM +0530, Girish Joshi via Libc-alpha wrote:
>>>> Hi Florian,
>>>> Thanks for the review.
>>>> Turns out `diff` was not working with the given input files, this
>>>> patch fixes that.
>>>> I'm not sure if there is a list of people who volunteered to document
>>>> some specific functions.
>>>> So that comment from the output html is removed.
>>>>
>>>> Thanks!
>>>> Girish Joshi
>>>>
>>>> On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
>>>>>
>>>>> * Girish Joshi via Libc-alpha:
>>>>>
>>>>>> Hello,
>>>>>> As per the Missing Functions section of master Todo list[1],
>>>>>> scripts/documented.sh needs to be updated.
>>>>>>
>>>>>> Could someone please review this patch?
>>>>>> It removes the reference to linuxthreads/*.taxi files from the script
>>>>>> and removes the individual email address from the output.
>>>>>
>>>>> The script still doesn't seem to work, probably because the symbol names
>>>>> are not extract properly:
>>>>>
>>>>> 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
>>>>> 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
>>>>> 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
>>>>>
>>>>> _dl_find_object is documented.
>>>>>
>>>>> _IO_vfscanf should be recognized as an alias of vfscanf, but this is
>>>>> more of an RFE than a bug fix.
>>>>>
>>>>> Thanks,
>>>>> Florian
>>>>>
>>>
>>>> From 5148c65ac07688dc5c3ee79a9b0f054232d64df3 Mon Sep 17 00:00:00 2001
>>>> From: Girish Joshi <girish946@gmail.com>
>>>> Date: Sun, 5 Mar 2023 11:15:00 +0530
>>>> Subject: [PATCH] manual: scripts/documented.sh updated
>>>>
>>>> References for linuxthreads are removed from scripts/documented.sh.
>>>> Also removed the individual email address from the html output.
>>>>
>>>> Fixed the steps to retrive undocumented functions
>>>> ---
>>>>  scripts/documented.sh | 44 +++++++++++++++++++++++++++++++------------
>>>>  1 file changed, 32 insertions(+), 12 deletions(-)
>>>>  mode change 100644 => 100755 scripts/documented.sh
>>>>
>>>> diff --git a/scripts/documented.sh b/scripts/documented.sh
>>>> old mode 100644
>>>> new mode 100755
>>>> index 7d1ffd4a92..dda6a32283
>>>> --- a/scripts/documented.sh
>>>> +++ b/scripts/documented.sh
>>>> @@ -1,9 +1,15 @@
>>>>  #!/bin/sh
>>>>  bindir=$1
>>>>
>>>> -VERSION=1.0
>>>> +# usage: scripts/documented.sh <path to glibc build directory>
>>>> +# for example:
>>>> +# scripts/documented.sh $HOME/build/glibc
>>>>
>>>> -grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
>>>> +VERSION=1.1
>>>> +
>>>> +# Collect the documented functions from manual/*.texi files.
>>>> +
>>>> +grep -E -h @deftypefu?nx? manual/*.texi |
>>>>  sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
>>>>  sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
>>>>  sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
>>>> @@ -12,9 +18,12 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
>>>>  sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
>>>>  sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
>>>>  sed -e '/^alloca$/d' |
>>>> +sed -r '/^\s*$/d' |
>>>>  sort -u > DOCUMENTED
>>>>
>>>> -nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
>>>> +
>>>> +# extract the exported symbols from the libraries.
>>>> +nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
>>>>  grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
>>>>  sed 's/\(@.*\)//' |
>>>>  cut -b 12- |
>>>
>>> You can change this to 'cut -v 20-' and avoid the extra
>>> AVAILABLE_WITHOUT_OFFSET part.
>>>
>>> Thanks,
>>> Joe
>>>> @@ -25,6 +34,9 @@ sed -e '/^_mcleanup$/d' -e '/^_rpc_dtablesize$/d' -e '/^_seterr_reply$/d' |
>>>>  sed -e '/^_nss/d' -e '/^_setjmp$/d' |
>>>>  sort -u > AVAILABLE
>>>>
>>>> +# remove offset details from nm output
>>>> +cat AVAILABLE | cut -b 9-  | sort -u  > AVAILABLE_WITHOUT_OFFSET
>>>> +
>>>>  cat <<EOF
>>>>  <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
>>>>  <html>
>>>> @@ -39,9 +51,7 @@ cat <<EOF
>>>>      which are not yet documented in the manual.  This list is
>>>>      automatically created and therefore might contain errors.  Please
>>>>      check the latest manual (available from the CVS archive) before
>>>> -    starting to work.  It might also be good to let me know in
>>>> -    advanace on which functions you intend to work to avoid
>>>> -    duplication.</p>
>>>> +    starting to work.</p>
>>>>
>>>>      <p>A few comments:</p>
>>>>
>>>> @@ -52,15 +62,23 @@ cat <<EOF
>>>>        <li>Similarly for the LFS functions (those ending in 64).</li>
>>>>      </ul>
>>>>
>>>> -    <p>The function sombody already volunteered to document are marked
>>>> -    with a reference to the person.</p>
>>>>
>>>>      <center><table>
>>>>  EOF
>>>>
>>>>  n=0
>>>> -diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
>>>> -expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
>>>> +
>>>> +# get the list of undocumented functions
>>>> +grep -Fvx -f DOCUMENTED AVAILABLE_WITHOUT_OFFSET > UNDOCUMENTED_WITHOUT_OFFSET
>>>> +
>>>> +# get undocumented functions with offset info
>>>> +for func in `cat UNDOCUMENTED_WITHOUT_OFFSET`
>>>> +do
>>>> +  grep " $func$" AVAILABLE >> UNDOCUMENTED
>>>> +done
>>>> +
>>>> +diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
>>>> +expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
>>>>  while read name; do
>>>>    line="$line
>>>>  <td><tt>$name</tt></td>"
>>>> @@ -95,8 +113,10 @@ cat <<EOF
>>>>      </table></center>
>>>>
>>>>      <hr>
>>>> -    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
>>>> -Generated on $(date) with documented.sh version $VERSION
>>>> +    Generated on $(date) with documented.sh version $VERSION
>>>>    </body>
>>>>  </html>
>>>>  EOF
>>>> +
>>>> +# cleanup the files
>>>> +rm -rf UNDOCUMENTED_WITHOUT_OFFSET AVAILABLE_WITHOUT_OFFSET AVAILABLE
>>>> --
>>>> 2.39.2
>>>>
>>>

The issues from previous reviews seems not to be present and the output is
an improvement over the current status. LGTM, thanks.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>

PS: next time please send the patch as inline as well (it is easier to
    comment). Also for newer version send a new message with a vX.