PPA submissions are shown in gibberish and are not added

Asked by RobertX

Because of Linux Mint 20's incompatibilities with Brother printer drivers, I was forced to return to Linux Mint 19.3. I opened the Cubic Wizard and started to remaster Linux 19.3 and once I added a repository, here was the message:

root@cubic:~# add-apt-repository ppa:libreoffice/ppa
Unable to init server: Could not connect: Connection refused
Unable to init server: Could not connect: Connection refused
You are about to add the following PPA:
Traceback (most recent call last):
  File "/usr/lib/linuxmint/mintSources/mintSources.py", line 1916, in <module>
    add_repository_via_cli(ppa_line, codename, "-y" in args, use_ppas)
  File "/usr/lib/linuxmint/mintSources/mintSources.py", line 157, in add_repository_via_cli
    print(" %s" % (ppa_info["description"]))
UnicodeEncodeError: 'ascii' codec can't encode character '\xa0' in position 712: ordinal not in range(128)
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 145, in apport_excepthook
    os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o640), 'wb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/var/crash/_usr_lib_linuxmint_mintSources_mintSources.py.0.crash'

Original exception was:
Traceback (most recent call last):
  File "/usr/lib/linuxmint/mintSources/mintSources.py", line 1916, in <module>
    add_repository_via_cli(ppa_line, codename, "-y" in args, use_ppas)
  File "/usr/lib/linuxmint/mintSources/mintSources.py", line 157, in add_repository_via_cli
    print(" %s" % (ppa_info["description"]))
UnicodeEncodeError: 'ascii' codec can't encode character '\xa0' in position 712: ordinal not in range(128)

Even the Cubic's PPA returned the same error.

I just started doing this just today, and when I checked, the PPA is not immediately added.

NOTE: Linux Mint 19.3 Tricia is based on Ubuntu's Bionic Beaver.

Question information

Language:
English Edit question
Status:
Solved
For:
Cubic Edit question
Assignee:
No assignee Edit question
Solved by:
RobertX
Solved:
Last query:
Last reply:
Revision history for this message
Cubic PPA (cubic-wizard) said :
#1

Which OS version are you running Cubic in?

On Cubic's terminal page, what do you get when you try to ping a well know URL. for example:

    $ ping google.com

Revision history for this message
RobertX (bobbyyu1) said :
#2

I have Linux Mint 20 MATE Ulyana installed, which is based on Ubuntu
Focal Fossa (20.4).

Here's what I got when I pinged Google.com:

PING google.com (172.217.1.14) 56(84) bytes of data.
64 bytes from yyz10s14-in-f14.1e100.net (172.217.1.14): icmp_seq=1
ttl=120 time=42.7 ms
64 bytes from yyz10s14-in-f14.1e100.net (172.217.1.14): icmp_seq=2
ttl=120 time=43.7 ms
64 bytes from yyz10s14-in-f14.1e100.net (172.217.1.14): icmp_seq=3
ttl=120 time=42.5 ms
64 bytes from yyz10s14-in-f14.1e100.net (172.217.1.14): icmp_seq=4
ttl=120 time=43.0 ms
64 bytes from yyz10s14-in-f14.1e100.net (172.217.1.14): icmp_seq=5
ttl=120 time=42.1 ms
64 bytes from yyz10s14-in-f14.1e100.net (172.217.1.14): icmp_seq=6
ttl=120 time=42.8 ms
64 bytes from yyz10s14-in-f14.1e100.net (172.217.1.14): icmp_seq=7
ttl=120 time=42.4 ms
64 bytes from yyz10s14-in-f14.1e100.net (172.217.1.14): icmp_seq=8
ttl=120 time=41.0 ms
64 bytes from yyz10s14-in-f14.1e100.net (172.217.1.14): icmp_seq=9
ttl=120 time=41.8 ms
64 bytes from yyz10s14-in-f14.1e100.net (172.217.1.14): icmp_seq=10
ttl=120 time=42.8 ms
^C
--- google.com ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9015ms
rtt min/avg/max/mdev = 41.050/42.515/43.710/0.711 ms

Thank you.

Revision history for this message
Cubic PPA (cubic-wizard) said :
#3

Ok. Your network access is working fine in Cubic.

Firstly, delete everything under '/var/crash/' since this is causing a secondary FileNotFoundError error.

The program that adds your PPAs is having a problem interpreting the characters in a PPA.

Did you cut-and-paste the command `add-apt-repository ppa:libreoffice/ppa`.
Perhaps there is a hidden non-ascii character in there?
Do you get the same result if you type the full command?

May be there is a stray character in some of the files for your existing PPAs?...
What files do you have in /etc/apt/sources.d/ ?
Have you added anything to /etc/apt/sources.list ?

A lot of programs require UTF-8 character encoding, instead of ASCII.
Did you change the character encoding of your customized environment in Cubic?

Inside Cubic, try running the commands from comment #2 from this question to ensure you have UTF-8 setup...

   https://answers.launchpad.net/cubic/+question/694179

Revision history for this message
RobertX (bobbyyu1) said :
#4

> Firstly, delete everything under '/var/crash/' since this is causing a
> secondary FileNotFoundError error.

There was no such directory, both in Linux Mint 20's installed file
system and Cubic's terminal.

> Did you cut-and-paste the command `add-apt-repository ppa:libreoffice/ppa`.
> Perhaps there is a hidden non-ascii character in there?
> Do you get the same result if you type the full command?

No; it is when I typed out the commands that I discovered the problem.

> May be there is a stray character in some of the files for your existing PPAs?...
> What files do you have in /etc/apt/sources.d/ ?
> Have you added anything to /etc/apt/sources.list ?

There were no such files/directories in those locations.

> A lot of programs require UTF-8 character encoding, instead of ASCII.
> Did you change the character encoding of your customized environment in Cubic?
>
> Inside Cubic, try running the commands from comment #2 from this
> question to ensure you have UTF-8 setup...
>
> https://answers.launchpad.net/cubic/+question/694179
>
This was the output of the commands entered:

root@cubic:/etc/apt# dpkg -s locales | grep 'Package\|Version'
Package: locales
Version: 2.27-3ubuntu1
root@cubic:/etc/apt# echo $LC_CTYPE

root@cubic:/etc/apt#  echo $LANG

root@cubic:/etc/apt# locale
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=
root@cubic:/etc/apt# locale charmap
ANSI_X3.4-1968
root@cubic:/etc/apt# cat /etc/default/locale
#  File generated by update-locale
root@cubic:/etc/apt# cat /etc/rc.conf
cat: /etc/rc.conf: No such file or directory
root@cubic:/etc/apt# cat /etc/rc.conf
cat: /etc/rc.conf: No such file or directory
root@cubic:/etc/apt# cat /etc/locale.conf
cat: /etc/locale.conf: No such file or directory
root@cubic:/etc/apt# grep "^[^#;]" /etc/locale.gen
root@cubic:/etc/apt# python3 --version
Python 3.6.8
root@cubic:/etc/apt# python3 -c 'import sys; print(sys.version_info);
print(sys.stdout.encoding)'
sys.version_info(major=3, minor=6, micro=8, releaselevel='final', serial=0)
ANSI_X3.4-1968

I also took the liberty of adding the commands that supposedly would
change the encoding to UTF-8 (comment #3), both to Mint's terminal and
Cubic's termainal; it didn't help.

Revision history for this message
RobertX (bobbyyu1) said :
#5

Also, I tried entering commands to install vim to view some files: here
is what was printed out:

root@cubic:~# apt-get install vim
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
  vim : Depends: vim-common (= 2:8.0.1453-1ubuntu1) but
2:8.0.1453-1ubuntu1.1 is to be installed
E: Unable to correct problems, you have held broken packages.

Revision history for this message
RobertX (bobbyyu1) said :
#6

root@cubic:~# apt-get install vim
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 vim : Depends: vim-common (= 2:8.0.1453-1ubuntu1) but 2:8.0.1453-1ubuntu1.1 is to be installed
E: Unable to correct problems, you have held broken packages.

Revision history for this message
RobertX (bobbyyu1) said :
#7

I'm sorry, I meant comment #4, not comment #3.

Revision history for this message
Cubic PPA (cubic-wizard) said :
#8

I assume your host system is UTF-8 capable, since Cubic is running.

However, the locale output inside Cubic in comment #4 looks unusual.
I suspect the add-repository program just can't handle your character encoding in your custom OS.
How did you end up with POSIX and ANSI_X3.4-1968? Did you install some packages that changes it from the default UTF-8 ?

By the way, I meant for you to check the file /etc/apt/sources.list and the directory /etc/apt/sources.list.d.
You must have both for apt to work.

Revision history for this message
RobertX (bobbyyu1) said :
#9

> However, the locale output inside Cubic in comment #4 looks unusual.
> I suspect the add-repository program just can't handle your character encoding in your custom OS.
> How did you end up with POSIX and ANSI_X3.4-1968? Did you install some packages that changes it from the default UTF-8 ?

None that I know of.

> By the way, I meant for you to check the file /etc/apt/sources.list and the directory /etc/apt/sources.list.d.
> You must have both for apt to work.
>
I have found the following in official-package-repositories.list, which
was inside sources.list.d:

# Do not edit this file manually, use Software Sources instead.

deb http://packages.linuxmint.com tricia main upstream import backport
#id:linuxmint_main

deb http://archive.ubuntu.com/ubuntu bionic main restricted universe
multiverse
deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted
universe multiverse
deb http://archive.ubuntu.com/ubuntu bionic-backports main restricted
universe multiverse

deb http://security.ubuntu.com/ubuntu/ bionic-security main restricted
universe multiverse

There was nothing new found in sources.list.

Revision history for this message
RobertX (bobbyyu1) said :
#10

I thought I should point out that if I was to use Cubic to remaster
Linux Mint 20 itself, using Linux Mint 20, none of the errors show up.

Just staying.

Awesome program, thanks!

--
This email has been checked for viruses by AVG.
https://www.avg.com

Revision history for this message
Cubic PPA (cubic-wizard) said :
#11

LOL!

It should work regardless of the Ubuntu based OS you are using and the Ubuntu based OS you are customizing.
(The only restriction is, Cubic will not run in anything below Ubuntu 18.04, but you could still customize an older version of Ubuntu in Cubic if you wanted to).

When I get a chance, I'll download Linux Mint 20 MATE Ulyana and run it in a virtual machine, and I'll download and try to customize Linux Mint 19.3 Tricia in the virtual machine.

Revision history for this message
RobertX (bobbyyu1) said :
#12

An addition, I used Cubic and Mint 19.3 to remaster 19.3, that failed (i.e. same problem).

Revision history for this message
RobertX (bobbyyu1) said :
#13

> When I get a chance, I'll download Linux Mint 20 MATE Ulyana and run it
> in a virtual machine, and I'll download and try to customize Linux Mint
> 19.3 Tricia in the virtual machine.
>
Please do; it would be a great deal helpful in that you know what I mean.

Anyhow, the reason why I would remaster an older version is that printer
drivers for Linux Mint 20 are not compatible because the guy who
develops Linux Mint, in his infinite wisdom, decided to drop 32-bit
libraries.

Normally, I wouldn't bother with an older version, since support will
wane away in time, but I have to be able to use the printer or my house
guest will kill me.

No pressure; if you can't or won't fix the issue, so be it. I should be
grateful that you are trying to help and I thank you for it.

Now, should I revert to Ubuntu, or would that be a cop-out?

Revision history for this message
RobertX (bobbyyu1) said :
#14

Just so you know, Linux Mint 19.2 Tricia also gives the same error.

Linux Mint 19.2 is also based on Ubuntu 18.04 Bionic Beaver. The first Mint OS that is based on Bionic is Linux Mint 19.0.

Revision history for this message
Cubic PPA (cubic-wizard) said :
#15

FYI, I haven't had a chance to look at this yet, and I will get to it as soon as I can.

Revision history for this message
RobertX (bobbyyu1) said :
#16

All right man, take your time.

--
This email has been checked for viruses by AVG.
https://www.avg.com

Revision history for this message
RobertX (bobbyyu1) said :
#17

And Merry Christmas everyone!

Stay safe, stay awesome, and have a good holiday!

--
This email has been checked for viruses by AVG.
https://www.avg.com

Revision history for this message
Cubic PPA (cubic-wizard) said :
#18

Merry Christmas to you as well!

I finally got to download linuxmint-19.3-xfce-64bit.iso.

I installed it and got the following information...

###############################################################################
# Installed Linux Mint 19.3
###############################################################################

    $ dpkg -s locales | grep 'Package\|Version'
    Package: locales
    Version: 2.27-3ubuntu1
    $ echo $LC_CTYPE

    $ echo $LANG
    en_US.UTF-8
    $ locale
    LANG=en_US.UTF-8
    LANGUAGE=en_US
    LC_CTYPE="en_US.UTF-8"
    LC_NUMERIC="en_US.UTF-8"
    LC_TIME="en_US.UTF-8"
    LC_COLLATE="en_US.UTF-8"
    LC_MONETARY="en_US.UTF-8"
    LC_MESSAGES="en_US.UTF-8"
    LC_PAPER="en_US.UTF-8"
    LC_NAME="en_US.UTF-8"
    LC_ADDRESS="en_US.UTF-8"
    LC_TELEPHONE="en_US.UTF-8"
    LC_MEASUREMENT="en_US.UTF-8"
    LC_IDENTIFICATION="en_US.UTF-8"
    LC_ALL=
    $ locale charmap
    UTF-8
    $ cat /etc/default/locale
    # File generated by update-locale
    LANG="en_US.UTF-8"
    $ cat /etc/rc.conf
    cat: /etc/rc.conf: No such file or directory
    $ cat /etc/locale.conf
    cat: /etc/locale.conf: No such file or directory
    $ grep "^[^#;]" /etc/locale.gen
    en_US.UTF-8 UTF-8
    $ python3 --version
    Python 3.6.8
    $ python3 -c 'import sys; print(sys.version_info); print(sys.stdout.encoding)'
    sys.version_info(major=3, minor=6, micro=8, releaselevel='final', serial=0)
    UTF-8

###############################################################################
# Inside Cubic's Terminal
###############################################################################

    You have entered the virtual environment.
    Host and machine ids are equal (775ccb91c8f84539a863dbbb02766c59): refusing to link journals
    $ dpkg -s locales | grep 'Package\|Version'
    Package: locales
    Version: 2.27-3ubuntu1
    $ echo $LC_CTYPE

    $ echo $LANG

    $ locale
    LANG=
    LANGUAGE=
    LC_CTYPE="POSIX"
    LC_NUMERIC="POSIX"
    LC_TIME="POSIX"
    LC_COLLATE="POSIX"
    LC_MONETARY="POSIX"
    LC_MESSAGES="POSIX"
    LC_PAPER="POSIX"
    LC_NAME="POSIX"
    LC_ADDRESS="POSIX"
    LC_TELEPHONE="POSIX"
    LC_MEASUREMENT="POSIX"
    LC_IDENTIFICATION="POSIX"
    LC_ALL=
    $ locale charmap
    ANSI_X3.4-1968
    $ cat /etc/default/locale
    # File generated by update-locale
    $ cat /etc/rc.conf
    cat: /etc/rc.conf: No such file or directory
    $ cat /etc/locale.conf
    cat: /etc/locale.conf: No such file or directory
    $ grep "^[^#;]" /etc/locale.gen
    $ python3 --version
    Python 3.6.8
    $ python3 -c 'import sys; print(sys.version_info); print(sys.stdout.encoding)'
    sys.version_info(major=3, minor=6, micro=8, releaselevel='final', serial=0)
    ANSI_X3.4-1968

Revision history for this message
Cubic PPA (cubic-wizard) said :
#19

SOLUTION:

(1)
View the current locale settings.

    dpkg -s locales | grep 'Package\|Version'
    echo $LC_CTYPE
    echo $LANG
    locale
    locale charmap
    cat /etc/default/locale
    cat /etc/rc.conf
    cat /etc/locale.conf
    grep "^[^#;]" /etc/locale.gen
    python3 --version
    python3 -c 'import sys; print(sys.version_info); print(sys.stdout.encoding)'

(2)
Generate the required locale...

    root@cubic:~# dpkg-reconfigure locales

On the first page, notice noting is selected, which explains why it defaulted to POSIX.

Select "en_US.UTF-8 UTF-8" on the first page:

    [ ] en_US ISO-8859-1
    [ ] en_US.ISO-8859-15 ISO-8859-15
    [*] en_US.UTF-8 UTF-8

Select "en_US.UTF-8" on the second page:

    en_US.UTF-8

Select "OK" to go to the next page.

(3)
If you were to re-execute the commands in step 1, you would notice not much has changed.
Most of the values are still "POSIX"

The only two differences are these:

    root@cubic:~# cat /etc/default/locale
    # File generated by update-locale
    LANG=en_US.UTF-8

    root@cubic:~# grep "^[^#;]" /etc/locale.gen
    en_US.UTF-8 UTF-8

However, this tells us that we have indeed generated the required locale.

(3)
Next set the locale:

    export LC_ALL="en_US.UTF-8"

(4)
Now, if re-execute the commands in step 1, you would notice not everything is set to "en_US.UTF-8".

    root@cubic:~# dpkg -s locales | grep 'Package\|Version'
    Package: locales
    Version: 2.27-3ubuntu1
    root@cubic:~# echo $LC_CTYPE

    root@cubic:~# echo $LANG

    root@cubic:~# locale
    LANG=
    LANGUAGE=
    LC_CTYPE="en_US.UTF-8"
    LC_NUMERIC="en_US.UTF-8"
    LC_TIME="en_US.UTF-8"
    LC_COLLATE="en_US.UTF-8"
    LC_MONETARY="en_US.UTF-8"
    LC_MESSAGES="en_US.UTF-8"
    LC_PAPER="en_US.UTF-8"
    LC_NAME="en_US.UTF-8"
    LC_ADDRESS="en_US.UTF-8"
    LC_TELEPHONE="en_US.UTF-8"
    LC_MEASUREMENT="en_US.UTF-8"
    LC_IDENTIFICATION="en_US.UTF-8"
    LC_ALL=en_US.UTF-8
    root@cubic:~# locale charmap
    UTF-8
    root@cubic:~# cat /etc/default/locale
    # File generated by update-locale
    LANG=en_US.UTF-8
    root@cubic:~# cat /etc/rc.conf
    cat: /etc/rc.conf: No such file or directory
    root@cubic:~# cat /etc/locale.conf
    cat: /etc/locale.conf: No such file or directory
    root@cubic:~# grep "^[^#;]" /etc/locale.gen
    en_US.UTF-8 UTF-8
    root@cubic:~# python3 --version
    Python 3.6.8
    root@cubic:~# python3 -c 'import sys; print(sys.version_info); print(sys.stdout.encoding)'
    sys.version_info(major=3, minor=6, micro=8, releaselevel='final', serial=0)
    UTF-8

(5)
However, every time you exit the virtual environment and re-enter it, your locale settings will revert to the previous "POSIX" values.

To remedy this, make the change permanent by add the LC_ALL environment variable to the .profile file:

    root@cubic:~# echo 'export LC_ALL="en_US.UTF-8"' | tee -a .profile

(6)
Finally, test.

    root@cubic:~# apt-add-repository ppa:cubic-wizard/release

Revision history for this message
Cubic PPA (cubic-wizard) said :
#20

* Step (4) should have read:

Now, if you re-execute the commands in step 1, you would notice everything is set to "en_US.UTF-8".

Revision history for this message
RobertX (bobbyyu1) said :
#21

Your instructions work! Thanks!

I'll keep these instructions safe.

Great work, thank you, and that will be all. :D

Revision history for this message
Cubic PPA (cubic-wizard) said :
#22

Great!
Please mark this question "Solved" (or the system will erase it automatically).

Revision history for this message
RobertX (bobbyyu1) said :
#23

OK. Problem marked as solved.