[v4] localedata: Avoid concurrently written locales in gen-locale.sh
Checks
| Context |
Check |
Description |
| redhat-pt-bot/TryBot-apply_patch |
success
|
Patch applied to master at the time it was sent
|
| linaro-tcwg-bot/tcwg_glibc_build--master-arm |
success
|
Build passed
|
| linaro-tcwg-bot/tcwg_glibc_build--master-aarch64 |
success
|
Build passed
|
| redhat-pt-bot/TryBot-32bit |
success
|
Build for i686
|
| linaro-tcwg-bot/tcwg_glibc_check--master-aarch64 |
fail
|
Test failed
|
| linaro-tcwg-bot/tcwg_glibc_check--master-arm |
fail
|
Patch failed to apply
|
Commit Message
There is no cross-directory exclusion of concurrent $(gen-locales)
usage. Parallel localedef calls can clobber locale data as it is
being loaded by tests.
With the separate staging areas, there are no peer directories,
so hard-linking no longer happens. The touch command is therefore
unnecessary.
---
v4: Drop --no-hard-links option because it is no longer required.
localedata/gen-locale.sh | 22 +++++++++++++++-------
1 file changed, 15 insertions(+), 7 deletions(-)
Comments
Florian Weimer <fweimer@redhat.com> writes:
> There is no cross-directory exclusion of concurrent $(gen-locales)
> usage. Parallel localedef calls can clobber locale data as it is
> being loaded by tests.
>
> With the separate staging areas, there are no peer directories,
> so hard-linking no longer happens. The touch command is therefore
> unnecessary.
Thanks.
Reviewed-by: Sam James <sam@gentoo.org>
>
> ---
> v4: Drop --no-hard-links option because it is no longer required.
> localedata/gen-locale.sh | 22 +++++++++++++++-------
> 1 file changed, 15 insertions(+), 7 deletions(-)
>
> diff --git a/localedata/gen-locale.sh b/localedata/gen-locale.sh
> index 4762c04284..c449efe125 100644
> --- a/localedata/gen-locale.sh
> +++ b/localedata/gen-locale.sh
> @@ -32,17 +32,25 @@ generate_locale ()
> out=$3
> flags=$4
> ret=0
> +
> + # Use a staging area to avoid writing to locales concurrently.
> + # While this process is running, $$ is sufficiently unique.
> + stage="${common_objpfx}localedata/gen-locale.$$.tmp"
> + mkdir "$stage" 2>/dev/null || true
> +
> ${localedef_before_env} ${run_program_env} I18NPATH=../localedata \
> ${localedef_after_env} $flags -f $charmap -i $input \
> - ${common_objpfx}localedata/$out || ret=$?
> - if [ $ret -eq 0 ]; then
> - # The makefile checks the timestamp of the LC_CTYPE file,
> - # but localedef won't have touched it if it was able to
> - # hard-link it to an existing file.
> - touch ${common_objpfx}localedata/$out/LC_CTYPE
> + $stage/$out || ret=$?
> + if [ $ret -eq 0 ] ; then
> + # Ignore errors in case some other process has created the same locale.
> + # This rename operation should be atomic, and it should fail if the
> + # $out locale already exists (rename fails with ENOTEMPTY).
> + mv $stage/$out ${common_objpfx}localedata/. 2>/dev/null || true
> + rm -rf $stage
> else
> + rm -rf $stage
> echo "Charmap: \"${charmap}\" Inputfile: \"${input}\"" \
> - "Outputdir: \"${out}\" failed"
> + "Outputdir: \"${out}\" failed (exit status $ret)"
> exit 1
> fi
> }
@@ -32,17 +32,25 @@ generate_locale ()
out=$3
flags=$4
ret=0
+
+ # Use a staging area to avoid writing to locales concurrently.
+ # While this process is running, $$ is sufficiently unique.
+ stage="${common_objpfx}localedata/gen-locale.$$.tmp"
+ mkdir "$stage" 2>/dev/null || true
+
${localedef_before_env} ${run_program_env} I18NPATH=../localedata \
${localedef_after_env} $flags -f $charmap -i $input \
- ${common_objpfx}localedata/$out || ret=$?
- if [ $ret -eq 0 ]; then
- # The makefile checks the timestamp of the LC_CTYPE file,
- # but localedef won't have touched it if it was able to
- # hard-link it to an existing file.
- touch ${common_objpfx}localedata/$out/LC_CTYPE
+ $stage/$out || ret=$?
+ if [ $ret -eq 0 ] ; then
+ # Ignore errors in case some other process has created the same locale.
+ # This rename operation should be atomic, and it should fail if the
+ # $out locale already exists (rename fails with ENOTEMPTY).
+ mv $stage/$out ${common_objpfx}localedata/. 2>/dev/null || true
+ rm -rf $stage
else
+ rm -rf $stage
echo "Charmap: \"${charmap}\" Inputfile: \"${input}\"" \
- "Outputdir: \"${out}\" failed"
+ "Outputdir: \"${out}\" failed (exit status $ret)"
exit 1
fi
}