Consider replacing mawk with gawk in main

Bug #1841654 reported by amano
38
This bug affects 6 people
Affects Status Importance Assigned to Milestone
mawk (Ubuntu)
Opinion
Undecided
Unassigned
ubuntu-mate-meta (Ubuntu)
Opinion
Undecided
Unassigned
ubuntu-meta (Ubuntu)
Opinion
Undecided
Unassigned

Bug Description

For POSIX compatibility reasons Ubuntu ships with mawk ("Mike's awk" = mawk) in main. There is an awk-symlink to mawk, thus mawk is the official awk implementation in Ubuntu.

== Reasons against keeping mawk ==

*The mawk package is synced from debian and it is heavily undermaintained: Debian (and thus Ubuntu) still ships version 1.3.3-17, the same version at least since oldoldstable: https://packages.debian.org/search?searchon=names&keywords=mawk

*maintainer Thomas E. Dickey (https://invisible-island.net/mawk/) called officially out that Debian "neglected" mawk in 2014 ("As noted, mawk has been neglected by some packagers (see for example this http://bugs.debian.org/cgi-bin/pkgreport.cgi?package=mawk)". And Debian (and Ubuntu) still ship the same version five years later

*Most other distributions ship mawk at least in its last official incarnation 1.3.4 in their repositories. Dickey lists AIX, Fedora, FinkPorts (Mac OS X), FreeBSD port, Gentoo, HPUX, MacPorts (Mac OS X), NetBSD pkgsrc/lang, OpenCSW (Solaris). That version is from 2014 but updated snapshots are released in regular intervals. The current snapshot is from February 2019: https://github.com/ThomasDickey/mawk-snapshots/blob/ab13a164013940e90c0b1ad36a039feae06a0b65/CHANGES

*A planned rewrite mawk2 was planned by original author Mike Brennan in 2016 but obviously came to nothing: https://github.com/ploxiln/mawk-2/commits/master

*Thus mawk sits in Ubuntu main in a version published upstream in 2009 and celebrates its 10th anniversary of negligence. In a state that Debian was called out for 5 years ago.

*This year it is 10 years unmaintained and largely untouched. At the end of the next LTS-support-cycle we can celebrate 15 years of not supporting it.

*awk is included in Linux for POSIX standard compliance. Mawk is known to be fast and small but it is the implementation that is farthest from being POSIX compliant, missing things like named character classes like [[:space:]] within its EREs.

== Reasons for gawk as a replacement ==

*It is the official GNU awk implementation and known to work well with the rest of the GNU userland.

*It is actively maintained with the last version 5.01 shipping in June, 2019 (even if Ubuntu will obviously still ship 4.2 for Eoan).

*It is mostly compliant with the POSIX standard.

*Most other distributions ship it as the standard POSIX implementation, with a awk symlink. So it had security reviews already by Red Hat and others.

== Possible problems with switching to gawk in time for Ubuntu 20.4 ==

- It might need to be MIRed by the Ubuntu security team and a review might take some time. So I filed this bug early with Eoan not yet out of the door.

- It is much larger than mawk, the Ubuntu package is 1600 kB in size, while mawk is only about 190KB. Thus some might want to split out some basic functionality to save size. Like what is vim-tiny to vim. To start gawk in --traditional or --posix mode and disable the extensions at compile time might be a good start to reduce the size. See: https://www.gnu.org/software/gawk/manual/html_node/Additional-Configuration-Options.html#Additional-Configuration-Options

=============================================================================

ProblemType: Bug
DistroRelease: Ubuntu 19.04
Package: mawk 1.3.3-17ubuntu3
ProcVersionSignature: Ubuntu 5.0.0-27.28-generic 5.0.21
Uname: Linux 5.0.0-27-generic x86_64
ApportVersion: 2.20.10-0ubuntu27.1
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
Date: Tue Aug 27 20:12:11 2019
Dependencies:
 gcc-9-base 9.1.0-2ubuntu2~19.04
 libc6 2.29-0ubuntu2
 libgcc1 1:9.1.0-2ubuntu2~19.04
 libidn2-0 2.0.5-1
 libunistring2 0.9.10-1ubuntu2
InstallationDate: Installed on 2018-02-23 (550 days ago)
InstallationMedia: Ubuntu 18.04 LTS "Bionic Beaver" - Alpha amd64 (20180214)
ProcEnviron:
 LANGUAGE=de_AT:de
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=de_AT.UTF-8
 SHELL=/bin/bash
SourcePackage: mawk
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
amano (jyaku) wrote :
Revision history for this message
amano (jyaku) wrote :

CCing Steve Langasek since he was the Debian mawk maintainer that Dickey called out in the link above.

description: updated
description: updated
description: updated
description: updated
amano (jyaku)
description: updated
amano (jyaku)
description: updated
description: updated
description: updated
description: updated
amano (jyaku)
description: updated
amano (jyaku)
description: updated
description: updated
Revision history for this message
Thomas Dickey (dickey-his) wrote : Re: [Bug 1841654] Re: Replace mawk with gawk in main

On Tue, Aug 27, 2019 at 10:15:39PM -0000, amano wrote:
> *Most other distributions ship mawk at least in its last official
> incarnation 1.3.4 in their repositories. Dickey lists AIX, Fedora,
> FinkPorts (Mac OS X), FreeBSD port, Gentoo, HPUX, MacPorts (Mac OS X),
> NetBSD pkgsrc/lang, OpenCSW (Solaris). But even that version is from
> 2014 and doesn't seem to be developed anymore:
> https://github.com/ThomasDickey/mawk-20140914/commits/master

That's inaccurate and misleading.

--
Thomas E. Dickey <email address hidden>
https://invisible-island.net
ftp://ftp.invisible-island.net

Bryce Harrington (bryce)
tags: added: server-next
Joshua Powers (powersj)
tags: removed: server-next
Revision history for this message
amano (jyaku) wrote : Re: Replace mawk with gawk in main

@Thomas Dickey:
Wow, you were fast to find that topic ;)

I didn't want to spread misleading information, so I guess that I misread or misinterpreted my findings and mawk is still developed and maintained by you? Which is great to know. I probably just couldn't find your active development branch. Mind sharing some information about it?

I don't care if Debian/Ubuntu shipped with an updated mawk version (much smaller and faster) but for shellscript compatibility between the likes of say Fedora and Ubuntu a switch to gawk might still be worth considering. Any input would be welcome.

The only thing which is certainly bad is the current situation with a version from 2009 is shipped in main without being maintained by anyone anymore.

summary: - Replace mawk with gawk in main
+ Consider replacing mawk with gawk in main
description: updated
amano (jyaku)
description: updated
Revision history for this message
Thomas Dickey (dickey-his) wrote :

I updated the webpage last week, to reduce confusion:

https://invisible-island.net/mawk/mawk.html

However, back to the stated issue: if the packager follows the same guidelines as previously stated (100% compatibility with the existing scripts, no effort whatsoever required of the packager), it's not likely that any change will be made.

Revision history for this message
amano (jyaku) wrote :

Updated the description to include links to your current snapshots. Sorry for not finding them in the first place.

description: updated
Revision history for this message
Thomas Dickey (dickey-his) wrote :

NetBSD pkgsrc has a more recent version of mawk than stated in this report:

http://pkgsrc.se/lang/mawk

2019-02-15) Updated to version: mawk-1.3.4.20171017

MacPorts has the 2019-02-03 version:

https://github.com/macports/macports-ports/blob/master/lang/mawk/Portfile

The report as worded implies that none of the various packages are newer than "1.3.4" by omitting most of the details about patchlevel. In practice, the age of packages says more about the ported platform than the programs which are provided on that platform.

Regarding OpenCSW, the same comments about old version apply to gawk:

https://www.opencsw.org/packages/gawk/

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in mawk (Ubuntu):
status: New → Confirmed
Revision history for this message
Dave Chiluk (chiluk) wrote :

We (Indeed) were recently hit by mawk posix non-compliance as well. For all the reasons stated above, I would also like to see mawk replaced with gawk in main. I'd also like to see the mawk dependencies in the ubuntu-meta packages replaced with gawk. For that reason I'm opening a separate target for this bug in ubuntu-meta.

tags: added: indeed
Changed in ubuntu-meta (Ubuntu):
status: New → Confirmed
Norbert (nrbrtx)
Changed in ubuntu-mate-meta (Ubuntu):
status: New → Confirmed
tags: added: eoan
tags: removed: amd64 indeed wayland-session
Revision history for this message
Erik Auerswald (auerswal) wrote :

I would prefer to have a current mawk in Ubuntu, but if that does not happen, replacing mawk with gawk would be appreciated.

Revision history for this message
amano (jyaku) wrote :

With mawk 1.3.4.20200120-1 being uploaded to Debian testing now I suggest to mark this bug here as invalid and just sync the new mawk version to Focal. There is a separate bug for just updating mawk https://bugs.launchpad.net/ubuntu/+source/mawk/+bug/1332114 Let's follow up there...

Changed in mawk (Ubuntu):
status: Confirmed → Opinion
Changed in ubuntu-mate-meta (Ubuntu):
status: Confirmed → Opinion
Changed in ubuntu-meta (Ubuntu):
status: Confirmed → Opinion
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.