RFC: Sort tarballs created by the src-release.sh script

Message ID 1e772307-1f10-6e1c-5ffb-8513a41db5ab@redhat.com
State New
Headers
Series RFC: Sort tarballs created by the src-release.sh script |

Commit Message

Nick Clifton Oct. 3, 2022, 2:40 p.m. UTC
  Hi Guys,

   [This appears to be getting slightly out of hand...]

> Not sure what "creation date" might mean here. Assuming the script is > (typically) run from a git tree, perhaps the commit date of the top> level commit on the branch would be best to use?
Except that a commit to the branch that does not affect something that
would go into the tarball would then result in a changed date.

We could use the src-release.sh file itself, like this:



Would that work ?

Cheers
   Nick
  

Comments

Andreas Schwab Oct. 3, 2022, 7:56 p.m. UTC | #1
On Okt 03 2022, Nick Clifton wrote:

> We could use the src-release.sh file itself, like this:

The timestamp of checked out files has no meaning, and is generally not
reproducible.

> diff --git a/src-release.sh b/src-release.sh
> index 079b545ae7c..de1f98a70bb 100755
> --- a/src-release.sh
> +++ b/src-release.sh
> @@ -184,9 +184,15 @@ do_tar()
>      ver=$2
>      echo "==> Making $package-$ver.tar"
>      rm -f $package-$ver.tar
> +    # The sort command and --mtime, --group and --owner options are
> +    # used in order to create consistent, reproducible tarballs.
> +    # BUILD_DATE is set to SOURCE_DATE_EPOCH if defined, or the
> +    # modification date of this file otherwise.  cf:
> +    # https://reproducible-builds.org/docs/source-date-epoch/
> +    BUILD_DATE="$(date --utc --date="@${SOURCE_DATE_EPOCH:-$(date -r src-release.sh +%s)}" +%Y-%m-%d)"
>      find $package-$ver -follow \( $CVS_NAMES \) -prune \
> -       -o -type f -print \
> -       | tar cTfh - $package-$ver.tar
> +       -o -type f -print | LC_ALL=C sort \
> +       | tar cTfh - $package-$ver.tar --mtime=$BUILD_DATE --group=0 --owner=0

That won't work, as --mtime=$BUILD_DATE is interpreted in the local time zone.
  
Jan Beulich Oct. 4, 2022, 7:10 a.m. UTC | #2
On 03.10.2022 16:40, Nick Clifton wrote:
> Hi Guys,
> 
>    [This appears to be getting slightly out of hand...]

It might seem like that, yes, but I guess that's not entirely unexpected
for a topic of that kind - different people have different expectations
and habits.

>> Not sure what "creation date" might mean here. Assuming the script is > (typically) run from a git tree, perhaps the commit date of the top> level commit on the branch would be best to use?
> Except that a commit to the branch that does not affect something that
> would go into the tarball would then result in a changed date.

Every commit should be considered to affect the tarball, imo, as such
effects could also be indirect. If you really wanted to go that route,
then perhaps an alternative would be to use the commit date of the
most recent commit touching bfd/version.m4.

Jan
  

Patch

diff --git a/src-release.sh b/src-release.sh
index 079b545ae7c..de1f98a70bb 100755
--- a/src-release.sh
+++ b/src-release.sh
@@ -184,9 +184,15 @@  do_tar()
      ver=$2
      echo "==> Making $package-$ver.tar"
      rm -f $package-$ver.tar
+    # The sort command and --mtime, --group and --owner options are
+    # used in order to create consistent, reproducible tarballs.
+    # BUILD_DATE is set to SOURCE_DATE_EPOCH if defined, or the
+    # modification date of this file otherwise.  cf:
+    # https://reproducible-builds.org/docs/source-date-epoch/
+    BUILD_DATE="$(date --utc --date="@${SOURCE_DATE_EPOCH:-$(date -r src-release.sh +%s)}" +%Y-%m-%d)"
      find $package-$ver -follow \( $CVS_NAMES \) -prune \
-       -o -type f -print \
-       | tar cTfh - $package-$ver.tar
+       -o -type f -print | LC_ALL=C sort \
+       | tar cTfh - $package-$ver.tar --mtime=$BUILD_DATE --group=0 --owner=0
  }

  # Compress the output with bzip2