[FFe]Please provide 'locales-all' as in Debian

Bug #1394929 reported by Sylvain Beucler
20
This bug affects 3 people
Affects Status Importance Assigned to Milestone
glibc (Ubuntu)
Fix Released
Undecided
Unassigned
langpack-locales (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

Repurposing this bug as a Feature Freeze exception request to merge langpack-locales into glibc, and make locales-all available in the process. Working on getting that all done and properly tested by tonight, but if this doesn't make it, it'll happen in early 16.10 instead.

[Original Report]
Hi,

Debian provides a 'locales-all' package, which is very convenient to ensure all locales are available, and installs very quickly.

It's particularly handy when providing i18n'd web applications (e.g. that use php-gettext).
As they may be accessed by visitors of all supported languages, rather than documenting how the server administrator should enable a list of 10-15 languages by reconfiguring 'locales', the packager can just add 'locales-all' to the Recommends or Depends.
See for instance the FusionForge package in Debian (https://packages.debian.org/jessie/gforge-web-apache2).

Cheers!
Sylvain

Revision history for this message
Thorsten Glaser (mirabilos) wrote :

I also require this package, see: http://askubuntu.com/q/435954/222799

Changed in glibc (Ubuntu):
status: New → Confirmed
Revision history for this message
Matthias Klose (doko) wrote :

the package is not required. you can generate the locales on the fly for package builds (or build-depending on various language-pack packages).

$ cat locale-gen
#!/bin/sh

LOCPATH=`pwd`/locales
export LOCPATH

[ -d $LOCPATH ] || mkdir -p $LOCPATH

umask 022

echo "Generating locales..."
while read locale charset; do
        case $locale in \#*) continue;; esac
        [ -n "$locale" -a -n "$charset" ] || continue
        echo -n " `echo $locale | sed 's/\([^.\@]*\).*/\1/'`"
        echo -n ".$charset"
        echo -n `echo $locale | sed 's/\([^\@]*\)\(\@.*\)*/\2/'`
        echo -n '...'
        if [ -f $LOCPATH/$locale ]; then
            input=$locale
        else
            input=`echo $locale | sed 's/\([^.]*\)[^@]*\(.*\)/\1\2/'`
        fi
        localedef -i $input -c -f $charset $LOCPATH/$locale #-A /etc/locale.alias
        echo ' done'; \
done <<EOF
de_DE ISO-8859-1
de_DE@euro ISO-8859-15
en_HK ISO-8859-1
en_US.UTF-8 UTF-8
es_ES ISO-8859-1
EOF

echo "Generation complete."

$ LOCPATH=... <run your command>

Revision history for this message
Sylvain Beucler (beuc) wrote :

Hi,

I agree that there's way to work-around this missing package, but:
- this is much slower, and
- the fact that "locales-all" is not available on Ubuntu requires the project to package a number of packages from Debian.
- also I wouldn't feel safe automating this work-around, for instance I've got cases where the admin just didn't want to install a particular locale due to incorrect translations. Having "locales-all" in Recommends: allowed the necessary flexibility.

Is there a particular reason why Ubuntu shouldn't ship this package?

Revision history for this message
Sylvain Beucler (beuc) wrote :

"requires the project to package a number of packages from Debian."
I meant requires to patch.

Revision history for this message
Thorsten Glaser (mirabilos) wrote :

Hi Doko,

the package *is* required if you programmatically need to have “all”, or at least a large set of, locales available.

Possible use case here is a package dependency (declaratively).

In the specific case of FusionForge, I have seen it (both the native tracker and the embedded mediawiki) malfunction if locales-all was not installed, even getting PHP to segfault.

Another use case is to ensure something is available to users, especially nōn-root users, in the default location (so there is no need to set LOCPATH. (This can help ssh, sudo, etc.)

Revision history for this message
Sylvain Beucler (beuc) wrote :

Hi,

Is there any progress on this?
You didn't answer me.. :/

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Martin Pitt just let me know on IRC that there is a plan to move locales in Ubuntu from the langpack-locales source package to the glibc source package. In connection with that, langpack-all would be made available in Ubuntu.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

s/langpack-all/locales-all/

Revision history for this message
Hans Joachim Desserud (hjd) wrote :

>there is a plan to move locales in Ubuntu from the langpack-locales source package to the glibc source package. In connection with that, langpack-all would be made available in Ubuntu.

Hi, do you have a rough time estimate on this transition?

I see from http://qa.ubuntuwire.com/ftbfs that at least groovy2 requires locales-all in order to build. I don't know whether it is strictly needed or if something like the workaround from #2 could be used instead.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

On 2014-12-27 17:56, Hans Joachim Desserud wrote:
> On 2014-12-04 16:10, Gunnar Hjalmarsson wrote:
>> there is a plan to move locales in Ubuntu from the langpack-locales
>> source package to the glibc source package. In connection with
>> that, locales-all would be made available in Ubuntu.
>
> Hi, do you have a rough time estimate on this transition?

No. My personal guess is that it's not a high priority thing.

Revision history for this message
Hans Joachim Desserud (hjd) wrote :

>> Hi, do you have a rough time estimate on this transition?

>No. My personal guess is that it's not a high priority thing.

I guessed the same, but thought I'd check. Thanks. :)

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in langpack-locales (Ubuntu):
status: New → Confirmed
Revision history for this message
Andrey Galkin (andvgal) wrote :

Also run into this in scope of automatic unified provisioning of Ubuntu and Debian systems.
Is it really a difficult task to import already existing Debian package?

Adam Conrad (adconrad)
summary: - Please provide 'locales-all' as in Debian
+ [FFe]Please provide 'locales-all' as in Debian
description: updated
Revision history for this message
Adam Conrad (adconrad) wrote :

Here's the diff between the 2.23 update (bug #1521172) and the locales merge proposed here.

Revision history for this message
Steve Langasek (vorlon) wrote :

@@ -96,7 +96,8 @@
 Section: localization
 Priority: standard
 Depends: libc-bin (>> 2.23), ${misc:Depends}, debconf | debconf-2.0
-Replaces: manpages-fr-extra (<< 20141022)
+Breaks: libc-bin (<< 2.23)
+Replaces: libc-bin (<< 2.23), manpages-fr-extra (<< 20141022)
 Build-Profiles: <!stage1>
 Description: GNU C Library: National Language (locale) data [support]
  Machine-readable data files, shared objects and programs used by the

This introduces a Breaks/Replaces between two packages in the minimal set (locales, libc-bin). This will warrant upgrade testing from both 14.04 and 15.10. Has any upgrade testing been done already?

The Breaks/Replaces from locales to libc-bin is definitely required, due to the move of the update-locale script.

I see the breaks relationship is also bidirectional, with libc6 Breaks: (locales << 2.23), which further emphasizes the need for upgrade testing. Why is the breaks in that direction required? Is this related to the drop of the locale-gen --purge call from libc.postinst?

Otherwise, everything in this looks good to me. It appears to be a faithful translation of the existing langpack handling code in the Ubuntu-specific package, and where there is delta it makes sense in terms of bugfixing of the existing behavior (e.g., properly splitting the handling of /etc/locale.gen vs. /var/lib/locales/supported.d, and fixing handling of langpacks to not modify contents of /usr/lib/locale, so that the set of available locales is always the union of the debconf-configured locales and the langpacks).

Revision history for this message
Adam Conrad (adconrad) wrote :

14:28 <infinity> Breaks loops are pretty friendly (compared to Conflicts loops), they just both deconfigure, both upgrade, and both configure.
14:28 <infinity> I tested from 2.21, and it went fine, I can't see why it would be worse from older versions.
14:29 <infinity> (The libc->Breaks locales relationship is standard in the Debian packaging to make sure the tools match libc, I was just adding it back)

tags: added: block-proposed
Revision history for this message
Adam Conrad (adconrad) wrote :

Adding the block-proposed tag until more upgrade testing from trusty has been done.

Revision history for this message
Steve Langasek (vorlon) wrote :

Discussed with Adam that upgrade testing can happen after the package has been uploaded to xenial-proposed, so that this doesn't need to block the official builds, and can happen in parallel with the autopkgtest testing.

So this just needs a blocking bug opened, to be closed only once the upgrade testing from both wily and trusty has been done.

Revision history for this message
Adam Conrad (adconrad) wrote :

Upgrade testing has been done, and looks good. Dropping the block-proposed tag.

tags: removed: block-proposed
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (41.1 KiB)

This bug was fixed in the package glibc - 2.23-0ubuntu1

---------------
glibc (2.23-0ubuntu1) xenial; urgency=medium

  * Merge with 2.23 from experimental (LP: #1521172), remaining changes:
    - Remove the libc6 recommends on libc6-i686, which we don't build.
    - Enable libc6{,-dev}-armel on armhf and libc6{-dev}-armhf on armel.
    - Ship update-locale and validlocale in /usr/sbin in libc-bin.
    - Don't build locales or locales-all in Ubuntu, we rely on langpacks.
    - Heavily mangle the way we do service restarting on major upgrades.
    - Build i386 variants as -march=i686, build amd64 with -O3, and build
      ppc64 variants (both 64-bit and 32-bit) with -O3 -fno-tree-vectorize.
    - Build generic i386/i686 flavour with -mno-tls-direct-seg-refs.
    - Drop the libc6-xen flavour, as the above change covers Xen's needs.
    - Enable systemtap support, which is currently disabled in Debian.
    - debian/patches/ubuntu/local-warn-bsd-source.diff: Don't warn on
      _BSD_SOURCE || _SVID_SOURCE, so we don't have to fix everything
      that might define them and build with -Werror, drop after xenial.
    - Don't build libc-l10n, its contents get stripped for language-packs.
    - debian/control.in/libc: Remove a Breaks on s390x against a binary
      NMU version of check that will never exist in the Ubuntu archive.
  * Drop libc-bin manpages Recommends to Suggests to keep it in standard.
  * Build with -fno-pie and -no-pie on s390x, upstream isn't PIE-ready.
  * debian/testsuite-xfail-debian.mk: XFAIL tst-malloc-thread-{fail,exit}.
  * debian/patches/any/cvs-netinet-in-4.2.diff: Dropped, merged upstream.
  * debian/patches/any/cvs-netinet-tcp-4.2.diff: Dropped, merged upstream.
  * debian/patches/any/cvs-selinux-nscd.diff: Dropped, merged upstream.
  * debian/patches/powerpc/local-tune-power8.diff: Dropped, we build for P8.
  * debian/patches/ubuntu/submitted-no-stack-backtrace.diff: Rebase patch.
  * debian/patches/ubuntu/unsubmitted-increase-dtv-surplus.diff: Dropped.

  * Evaluate and merge/drop/rebase localedata patches from langpack-locales.
    - debian-*patch: Drop all langpack-locales patches copied from Debian.
    - ubuntu/localedata/bn-language.patch: Rebase for glibc.
    - ubuntu/localedata/ce_RU-new_locale.patch: Drop, merged upstream.
    - ubuntu/localedata/ckb_IQ-new_locale.patch: Rebase for glibc.
    - ubuntu/localedata/de_AT-february.patch: Drop, merged upstream.
    - ubuntu/localedata/en_CA-dateformat.patch: Rebase for glibc.
    - ubuntu/localedata/en_IE-am_pm.patch: Rebase for glibc.
    - ubuntu/localedata/eo_US.patch: Rebase for glibc.
    - ubuntu/localedata/es-decimal_point-thousands_sep.patch: Drop, merged.
    - ubuntu/localedata/es_DO-am_pm.patch: Rebase for glibc.
    - ubuntu/localedata/es_EV.patch: Drop, merged upstream.
    - ubuntu/localedata/gd_GB-days-months.patch: Drop, merged upstream.
    - ubuntu/localedata/it_IT-thousands_sep.patch: Drop, also in Debian.
    - ubuntu/localedata/ln_CD-new_locale.patch: Rebase for glibc.
    - ubuntu/localedata/lt_LT-currency.patch: Drop, merged upstream.
    - ubuntu/localedata/lv_LV-currency.patch: Rebase for glibc.
    - ubuntu/localedata/pt_BR-d_fmt.patch: ...

Changed in glibc (Ubuntu):
status: Confirmed → Fix Released
Adam Conrad (adconrad)
Changed in langpack-locales (Ubuntu):
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.