update-apt-xapian-index uses too much CPU and memory

On my old laptop, the CPU usage is not too bad (5%) but it uses insane amounts of memory, ca 100 MB. Most annoying thing since the dreaded scrollkeeper-update.

I confirm that too. It uses 98% CPU but for a relatively short period of time.

It uses all the available cpu and interrputs the work of people for no reason. Would someone kindly assign a priority to this bug?

Just happened to me on an Asus EEE 901. I was copying large files from a pendrive, and after few minutes my desktop was almost frozen, unusable. The system responded for a mouse action in 30-60 seconds. The hard drive led (SSD in fact) was continuously on. I managed to switch to vt1 and log in (it took a minute), and ran top. It showed that update-apt-xapi was using 100% CPU. After about 8-10 minutes of struggling, the process finished it's mysterious work, and the system was back to normal again. But rendering the box unusable for minutes can be frustrating. At least can somebody tell me why we need this background process at all? Can I uninstall it, without breaking functionality?

Il giorno gio, 28/05/2009 alle 19.06 +0000, Pelládi Gábor ha scritto:
> At least
> can somebody tell me why we need this background process at all? Can I
> uninstall it, without breaking functionality?

The program is described in apt-cache show as supporting the index of
packages that you see when you use "add/remove" from the applications
menu in the gnome panel. I uninstalled it and nothing bad happened (I
use synaptic or apt to install/uninstall applications).

"At least can somebody tell me why we need this background process at all"

I think it trawls through the titles and descriptions of your apt database of packages making it possible for you to search for packages not just for their title but also by their description. However I think in the past I have been able to search by package descriptions long before this process was crippling my PC.

Anyway my solution is a little less dramatic than uninstalling the thing (if its existence is necessary I don't know, I doubt it but have kept it in case). I just went into the directory /etc/cron.weekly and moved the file apt-xapian-index to the folder /etc/cron.monthly. It still rears its ugly head occasionally but at just six times between releases I can live with it.

Confirmed problem on Ubuntu 9.04 x86_64 Jaunty. I am running in Intel Core 2 Quad Q9650 CPU, and the update-apt-xapian process takes 100% of one core. As of this writing, it has run for 468:50:30 according to top with no end in sight. This is absurd.

I tried purging the package and killing the process, but then the Quick Search window in Synaptic no longer works. I tried making the /etc/cron.weekly/apt-xapian-index script non-executable, but it runs anyway.

Other than the Quick Search in Synaptic, I see no use for this process or its huge database. It either need to be fixed so that it doesn't eat 100% of a CPU core, or eliminated as a requirement for Synaptic's Quick Search so that it can be removed from the distribution.

I just killed the process I wrote about this afternoon. In checking the database, I noticed that it was last written over 2.5 hours ago. Yet, the update-apt-xapian process was still taking 100% of one CPU core until I just killed it. I took David's advice and moved the executing script from /etc/cron.weekly to /etc/cron.monthly. This needs to be fixed.

It eats up a very high percentage of memory on my machine. See attachment. I removed the package, seems like it's not very useful anyway.

I have an old 866Mhz machine with 256MB RAM.
Top only shows this process consuming around 40% of the CPU.
However, this process brings my machine to its knees.
The mouse doesn't move well. Menus take forever to pop-up.
Programs won't close without the warning "Application is not responding".
I think perhaps I am out of RAM and thus thrashing the cache.

I don't know how the kernel handles a background task wanting all available memory, thus leaving none for the application in focus.

It takes around 10 to 20 minutes to complete running on my machine.

I am running Ubuntu 8.10

This bug was fixed in the package apt-xapian-index - 0.21ubuntu2

---------------
apt-xapian-index (0.21ubuntu2) karmic; urgency=low

  * run the weekly update with nice and ionice (LP: #363695)

 -- Michael Vogt <email address hidden> Tue, 14 Jul 2009 09:22:34 +0200

I made it run with nice and ionice now. That should make it somewhat less problematic. I'm testing the memory behavior now in a virtual machine with little resources to see what can be done here.

> I made it run with nice and ionice now.

That sounds like a band-aid solution. This must be broken either in code or concept if it consumes so much resources. Why doesn't each package installation add itself to a database, instead of having an indexing monster run regularly even if no new packages are installed?

Is the "fix" committed just in Karmic? It needs to be fixed in other releases since that's where we all encountered it.

Any changes I made to the status of this bug where accidental. Currently update-apt-xapian-index is running and was making my system extremely sluggish with lots of IOWait, and my mouse clicked the Fix Released button on accident.

Considering the Add/Remove programs doesn't have much of a "live search" feel to it (when I search for packages it doesn't update for a few seconds and takes a lot of CPU) I don't see what this index is accomplishing.

As someone already mentioned, why can't this just be a done at package install/removal time? The prelink package does this. People expect the system to be under heavy usage while doing package management, but a daemon either needs to be unobtrusive and quick.

Also from a scalability point of view I imagine that this daemon redundantly touches a lot of packages that haven't changed, since it's execution time seems to (understandably) grow with the amount of packages I have installed. A post-install solution would cut down on the overall time wasted creating this index.

This is still a problem with:
0.21ubuntu2

Firefox and tracker are bad enough but this is ridicules.

Confirming #18 Olafur Arason's experience..

On Karmic, with apt-xapian-index-0.21ubuntu2

Agreed with #14 Tormod Volden, and I really wish it was even a band-aid solution.

Test-bed:
Pentium-III 500Mhz with 192MB RAM running a minimal install and a light-weight LXDE/openbox desktop, idle, with only 48MB of the 192MB physical RAM being used.

Result: update-apt-xapian-index consumes 92-98% CPU, and over 37.5% of physical RAM.

I'm sorry, but I do not think the "run with nice and ionice" could be considered a proper fix. So if you'll kindly allow me, I'll change the bug status back to "confirmed" for now.

85.4% CPU - Intel 2.4Ghz/1G Ram.
Jaunty 2.6.28-16-generic (Gnome)

$ apt-cache policy apt-xapian-index
apt-xapian-index:
  Installed: 0.16
  Candidate: 0.16
  Version table:
 *** 0.16 0
        500 http://archive.ubuntu.com jaunty/main Packages
        100 /var/lib/dpkg/status

On Ubuntu 9.10, it accumulates a whole minute of CPU (observed with top), on a machine with dual-core 3GHz and 2GB RAM.

is this bug still present in 9.10 karmic ?

Mirza: Please read the other posts before adding comments. #21 is clearly still affected by this issue and Yes it takes waaaay too much system resources to update its index. On my AMD 3700+ it uses 100% CPU for at least 20s. (I'll monitor it the next time it runs in Karmic to get more exact time.)

Is it only Synaptic that uses this index? Does anyone know? Does Software Center use it as well? I mean, a non-geek person uses Software Center and the ultimate geek uses apt-get. This massive indexing affects all users but how many need or use this feature if it's only Synaptics Quicksearch that takes benefit from it.

The general search function in Synaptic does its job. The user types in a word, clicks enter and expects its CPU to work in order to find the target. Isn't the purpose of an indexing service to speed up the searching? It does, BUT, right now it's almost like its just MOVING the time when the CPU must work, FROM that time when the user expects it to work (when the user wants to search) TO that time when the user maybe watches a movie or playing a game? At least update-apt-xapian-index can in Karmic still bounce up while surfing the net, making the computer quite sluggish.

Running fully updated karmic and am still affected by this. 1 core sits at 100% for minutes

Not trying to cause a "me too" storm or anything, but since I promised to mention more exactly for how long it uses my CPU I'll add yet another comment. On AMD 3700+ it uses maximum resources for 40-60s. The situation is better now than in Jaunty where it would consume resources on almost every boot while in Karmic it runs more seldom.

Still bad though when I was running Gimp, Word 2007 through Wine and Firefox that this suddenly bounces up and makes everything sluggish :S If it has to run, run when computer is not in use.

Me too! ;-) Couldn't this be launched with a lower nice value?

update-apt-xapian-index

In my Toshiba One Core notebook uses 100% cpu for several minutes and started at any time making the machine unusable,

I'm experiencing the same problem on a Dell Laptop.

From the update-apt-xapian-index manpage:
> -u, --update
> incremental update, reindexing only those packages whose version has changed since the last run

Sounds reasonable, but is not used in /etc/cron.weekly/apt-xapian-index.

The attached patch adds this option.

MarcS, I suppose unified diff format is the standard nowadays (diff -u:)

Oh, my bad, excuse the newbie.
It's kinda trivial anyway.

Still present in up to date karmic Feb 3, 2010. Going to proceed with patch.

was experiencing this on lucid lynx alpha2

affects my too on Lucid Lynx, alpha 2 with all updates, running as guest in virtual box

Affects me too, starting to get rather annoying :(

I've been experiencing this at least since Karmic. On armel especially, it can sometimes cause sluggish performance for a few minutes after boot.

I have just switched to Kubuntu 9.10 from the Ubuntu Gnome distros, only to find it is all locked up until I kill this xapi process. It's pretty serious - completely locking up my computer for ages (until I kill it!).
I am amazed reading these comments it has been around so long and still hasn't been dealt with. I'm a bit annoyed because i finally persuaded my Dad (a fervent Windows man) to give Linux a go and gave him the Kubuntu disc I'd just written. D'oh.

Why do I think he's actually going to love this?

I too am plagued by this bug. My system locks up and my fans start spinning full-powered when this process runs. Perhaps defaulting it to a lower-priority nice level would remedy the issue?

In Jaunty, I found an interesting piece of code in /etc/cron.daily/mlocate. It checks if the laptop is on AC power, and if not, it exits without updating the locate database. I know this is not closely related to this bug, but apt-xapian-index doesn't seem to have this check. Maybe someone who has a laptop should run some tests to see if it worth putting into the script and could file a bug report if necessary. (I've never had a laptop, so I can't experiment with it).

running 9.10 on a 500mHz P3 notebook, this application pretty much shuts it down for a couple of hours on saturday morning. My experience is same as others I read here. I figured out what it was, killed it and saw the disk-banging stop and got my cpu back.

If all this does is give Synaptic Package Manager quick-search a current index - at least on saturday efternoon, it has to be the silliest resident ap I've ever seen.

I'm taking it out of cron.weekly, making it available to be run manually, while I'm on vacation and seeing if that lets it do whatever it seems to be doing.

If the wise folks who watch the gates on what can get into a Linux installation would keep an eye out for stuff like this, some of use with older machines would be better served.

To think that the Linux believers think that Windows is the only system with bloatware. This is a prime example.

I'm seeing the same problem on my fully patched Lucid Kubuntu box (P4, 512 Mb of RAM). Specifically, using 'atop' reveals system crippling levels of paging and disk reads - obviously the bottleneck on this system is memory rather than processor.

That said, my problem is not with the cron.weekly entry - there is a call to update-apt-xapian-index within my cron.daily/apt which seems to apply after each automatic package list update.

I'm highly tempted to comment out both the weekly and daily lines or, perhaps, to unset the excutable bit on
update-apt-xapian-index (since this would cause both scripts to skip the call).

At least I'm not contemplating the sanity-sap that would be looking at the source code yet...

i can confirm that this problem occurs in lucid version of distributions Kubuntu and Xubuntu.
it really sucks.

to make it better:
the relevant line 8 of /etc/cron.weekly/apt-xapian-index is (where $CMD is update-apt-xapian-index ):

 nice ionice -c3 $CMD --quiet

we can have much more than this:
- give a nice level of 19 instead of default 10
- enter a missing space so ionice will do its job
- make the indexer update only NEW packages instead of the whole database (as suggested by MarcS in comment 31)

this will get the most silence without dropping the indexer altogether:
- tested that the indexer will silently exit if database is already built
- tested that a 800mhz/256mb xcfe box stays responsive while indexing with the new nice/ionice pattern
which fixes this issue for me.

and the plain file for those who suffer and need a quick fix.

Implemented aexl's fix and problem gone with no problems so far.
Ran it from the shell replacing --quiet with --verbose and adding the --update option and now the described freeze disappeared because of faster and more reasonable execution time.
This was very annoying since I'm in a 512 Mb box and started paging like crazy and rendered unusable for several minutes an otherwise usable desktop.
I cat'd first the cron.weekly/apt-xapian-index file and did not included the --update option. I'm on Karmic.
Thanks a lot aexl!!

(Fix committed means the fix will appear in the Ubuntu repositories)

aexl, if the "-c3" -> "-c 3" change makes a difference, that's definitely a typo that should be straight-forward to get fixed. I would encourage you to check out https://wiki.ubuntu.com/SponsorshipProcess for how to get your contribution reviewed and released.

thank you tormod for pointing this out!

committed my patches upstream:
http://git.debian.org/?p=collab-maint/apt-xapian-index.git;a=commitdiff;h=d8cc0bd4468ba7144b8f610016422d9bd1471be6
http://git.debian.org/?p=collab-maint/apt-xapian-index.git;a=commitdiff;h=bccade9242a683ba09db00be8c55a2800b617a07

setting this now to fix committed.
(at least i see this appropriate when reading http://blog.launchpad.net/general/of-bugs-and-statuses
"For distribution packages, this status is intended to communicate that the developer has produced a fix in a location other than the official Ubuntu archive")

alright? can the show move on this way?

aexl, excellent! If you or someone else want to try to get this into Lucid as well, see https://wiki.ubuntu.com/StableReleaseUpdates

The meaning of "fix committed" is not perfectly coherent over all packages and teams, but I see it largely as "the fix will end up in the next Ubuntu version one way or another without anyone having to do anything more about it" which I think is the case here.

thank you tormod for your valuable information - i don't think i manage to do this soon so i'm stepping sidewards for whoever steps forward!

I don't think the space makes a difference:
Without the space:
ionice -p 3477
idle

with the space:
ionice -p 3525
idle

some commands don't really need a space for values:
ps aux|head -n1
is the same as
ps aux|head -n 1

* 2010/5/12 lavinog <email address hidden>:
>
> I don't think the space makes a difference:
> Without the space:
> ionice -p 3477
> idle

Here you've left the space.

> with the space:
> ionice -p 3525
> idle

The name of option which is said to have a difference is "-c", not
"-p". That surely may be bug in "ionice". You should take a look at
its documentation and the source :)

In such case patches by aexl would be just a workaround for a real bug.

S.

hey there,

i dont understand the aim of the discussion. when i do a patch then i dont rely on undocumented behaviour - it might change anytime.

best
aexl

* 2010/5/12 aexl wrote:
>
> i dont understand the aim of the discussion.

I and lavinog were discussing whether is there a bug in ionice
parameter's parsing :)

> when i do a patch then i dont rely on undocumented behaviour -
> it might change anytime.

aexl, that's OK :) and thank you for you contribution.

What is interesting for me -- whether such parsing of option's
parameter isn't really undocumented? That's not a criticism -- just
pure curiousity :)

S.

This bug was fixed in the package apt-xapian-index - 0.37

---------------
apt-xapian-index (0.37) unstable; urgency=low

  * Move #DEBHELPER# at the beginning of postinst, otherwise
    update-python-modules -p doesn't seem to always work. Closes: #581811

apt-xapian-index (0.36) unstable; urgency=low

  * Do not use ionice in cron job inside virtual environments.
    Patch by Raoul Bhatia. Closes: #581930.
  * Removed leftover debugging print when python-xdg is not available.
    Closes: #581906
  * Do not require a password for a simple update-apt-xapian-index run via
    dbus. Patch from Ubuntu by Michael Vogt. Closes: #582428.

apt-xapian-index (0.35) unstable; urgency=low

  * Tolerate (and if --verbose, report) .desktop file with invalid popcon
    fields
  * Added missing import. Closes: #581736
  * Run update-python-modules -p before updating the index in postinst.
    Closes: #581811

apt-xapian-index (0.34) unstable; urgency=low

  * Added aliases plugin, to feed synonims to the index
  * Tolerate older versions of python-debian
  * Give a nicer error message if run with not enough permissions
  * Added acknowledgements file mentioning sponsorship by the Fuss project

apt-xapian-index (0.33) unstable; urgency=low

  * Added missing import, fixing indexing of multilanguage descriptions

apt-xapian-index (0.32) unstable; urgency=low

  * Tolerate plugins' init functions that do not expect any arguments

apt-xapian-index (0.31) unstable; urgency=low

  [ David Paleino ]
  * debian/rules: set COLUMNS envvar when calling help2man (Closes: #577525)
  * debian/cron.weekly:
    - pass --update to update-apt-xapian-index, try to be less
      invasive during background runs (LP: #363695)
    - don't run the indexer when on battery power
  * axi-cache, update-xapian-index: update version number

  [ Axel Rutz ]
  * debian/cron.weekly:
    - inserted missing space in 'ionice -c 3'
    - give the indexer maximum niceness (LP: #363695)

  [ Enrico Zini ]
  * Switch to debhelper7 and distutils
  * Reorganised code in modules
  * Added a test suite
  * Added indexer for app-install .desktop file information

apt-xapian-index (0.30) unstable; urgency=low

  [ Enrico Zini ]
  * axi-cache: fix behaviour of again with no parameters
  * axi-cache: AND terms by default instead of OR
  * axi-cache: remove AND, OR and NOT from partial expressions when providing
    tab completion candidates
  * axi-cache: suggest tags from a preset list of facets when completing
    "axi-cache search "
  * axi-cache: when completing "axi-cache again " suggest context-sensitive
    terms from the previous "search" query
  * axi-cache: implemented showpkg
  * axi-cache: implemented showsrc
  * axi-cache: implemented depends, rdepends, policy, madison

  [ David Paleino ]
  * axi-cache.sh: move common "else" clause out of the last case..esac

apt-xapian-index (0.29) unstable; urgency=low

  * axi-cache: don't die horribly if a package exists in a-x-i but not in apt

apt-xapian-index (0.28) unstable; urgency=low

  * Added Homepage: field
  * Implemented axi-cache show and David provided its completion
  * Allow to run via dbus (Thanks to Michael Vogt)

apt-xapian-index (0....

I have a triple-core Phenom. It locked the system and wouldn't allow me to do anything. It logged me out of Xchat sessions.

7201 root 30 10 180m 139m 1172 R 93 3.8 0:35.63 update-apt-xapi <- line from top, after it let me type top.

It took ten or so minutes.

I was really pretty scared that my drive was going bad. . and it's less than a year old.

chalcedony, take a look one post above and you'll read:

> This bug was fixed in the package apt-xapian-index - 0.37
  ...
> ** Changed in: apt-xapian-index (Ubuntu)
> Status: Fix Committed => Fix Released

So there is nothing to do in this bugreport already, IMHO. You should just upgrade.

BTW, the new Maverick version seems to install without problems in Lucid, just get apt-xapian-index_0.37_all.deb from https://launchpad.net/ubuntu/+source/apt-xapian-index

s2,

bash: alias: --portrait: not found
llhull@Marcus:/$ sudo apt-get install apt-xapian-index
[sudo] password for llhull:
Reading package lists... Done
Building dependency tree
Reading state information... Done
apt-xapian-index is already the newest version.
The following packages were automatically installed and are no longer required:
  libdns45
Use 'apt-get autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 29 not upgraded.
llhull@Marcus:/$

Maybe I misunderstood you. Are you saying upgrade ubuntu? Can you be more explicit about what I need to do?

Linda

--- On Thu, 6/10/10, s2 <email address hidden> wrote:

> From: s2 <email address hidden>
> Subject: [Bug 363695] Re: update-apt-xapian-index uses too much CPU
> To: <email address hidden>
> Date: Thursday, June 10, 2010, 10:45 AM
> chalcedony, take a look one post
> above and you'll read:
>
> > This bug was fixed in the package apt-xapian-index -
> 0.37
>   ...
> > ** Changed in: apt-xapian-index (Ubuntu)
> >       Status: Fix Committed
> => Fix Released
>
> So there is nothing to do in this bugreport already,
> IMHO.  You should
> just upgrade.
>
> --
> update-apt-xapian-index uses too much CPU
> https://bugs.launchpad.net/bugs/363695
> You received this bug notification because you are a direct
> subscriber
> of the bug.
>
> Status in “apt-xapian-index” package in Ubuntu: Fix
> Released
>
> Bug description:
> Binary package hint: apt-xapian-index
>
> A background silent update process shouldn't use 98% CPU.
> It makes system sluggish with no visible reason.
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu/+source/apt-xapian-index/+bug/363695/+subscribe
>

Linda, well, I said this theoretically -- just out of deduction. But Tormod Volden gave the ground to my statement -- he directed to where you can get the exact binary package file:
https://launchpad.net/ubuntu/+source/apt-xapian-index/0.37/+build/1770663

> Are you saying upgrade ubuntu?

When the package is in new Ubuntu version, then yes, it would do too :). Not exactly now, as far as I can see.

Saulius.

Even with 0.37 I am not sure if this bug is really fixed. It is not hammering the hard drive like before, but it is still using 2 min of CPU time (and 100 MB RAM) here.

I confirm the observations of Tormod Volden. Should we reopen the bug or file new report.

tormod,

the first possible aim "make apt-xapian background indexing as nice as possible" is reached.

of course one could say "make apt-xapian background indexing an option which can be switched off" - i think here we need an analysis

* how can it be switched off
* what are the implications (like: do we only lose a fancy search option or does the whole apt system break)

any insights or weblinks welcome.

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

3004 root 30 10 177m 107m 5288 R 100 5.4 0:27.57 update-apt-xapi
 2359 tx 20 0 460m 62m 23m S 8 3.1 1:38.06 amule
    7 root 20 0 0 0 0 S 2 0.0 0:00.20 ksoftirqd/1
 2185 tx 20 0 336m 39m 8708 S 2 2.0 0:12.35 compiz
 3007 tx 20 0 19348 1360 940 R 2 0.1 0:00.02 top
    1 root 20 0 23784 1976 1288 S 0 0.1 0:00.71 init
    2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd

I think we can keep this bug report closed. The issue of apt-xapian-index paralyzing the machine has been solved. Many issues were identified here and solved by aexl and others. So please file new bug reports for further issues.

For apt-xapian-index taking 100-200 MB and keep running at full CPU for several minutes on our machines I think there are two issues: 1) It is not engineered in a resource-friendly fashion. I think it uses way to much for what it achieves. This is an upstream issue. 2) It is totally overkill and does not belong by default on the Ubuntu desktop. In my ignorance I don't know of any feature that it brings (except maybe collecting "popcon" statistics). But I should take this elsewhere than in this closed bug report.

Still an issue here too: Lucid updated to 22 June 2010, this office machine:

cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Pentium(R) 4 CPU 3.00GHz
stepping : 3
cpu MHz : 3000.000
cache size : 2048 KB

goes into 'lift-off' mode while updating the apt database via update-apt-xapian - I see no good reason for this, and certainly isn't a desirable activity for netbook/other low power devices..

Sorry, but this bug is *not* fixed. The proposed "fix" in Commment #54:
"This bug was fixed in the package apt-xapian-index - 0.37"
does not apply to any currently released Ubuntu & only is available in an Alpha (Active Development) system.
10.14 is the current LTS release & according to:
https://launchpad.net/ubuntu/+source/apt-xapian-index
only 0.25ubuntu2 is available for 10.04.

$ apt-cache policy apt-xapian-index
apt-xapian-index:
  Installed: 0.25ubuntu2
  Candidate: 0.25ubuntu2
  Version table:
 *** 0.25ubuntu2 0
        500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages
        100 /var/lib/dpkg/status
$ uname -a
[snipped] 2.6.32-23-generic #37-Ubuntu SMP Fri Jun 11 07:54:58 UTC 2010 i686 GNU/Linux

CPU usage (daily) is approx 97-98%.

Please reopen this bug, will actual testing, for Lucid. Users should not be expected to install pre-released alpha packages to resolve an LTS issue.

Added info: On a 2.4Ghz/1Gb machine:
$ sudo update-apt-xapian-index
and watching via top
sudo update-apt-xapian-index
  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
 4075 root 20 0 97468 78m 4604 R 95.4 7.8 0:28.02 update-apt-xapi
CPU is at 95.4%

From a 2.1Ghz/3Gb laptop:
  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
 2104 root 20 0 126m 65m 5156 R 99 2.2 0:07.28 update-apt-xapi
CPU is at 99%

NoOp, please see https://wiki.ubuntu.com/Bugs/Status for an explanation of bug status, and https://wiki.ubuntu.com/StableReleaseUpdates if you want to help make it fixed in Lucid.

So I was wondering why upgrading my media rig from hardy to karmic my mplayer kept pausing in the middle of films. It took me ages trying to catch the sod by alt-tabbing out and running top. This is one annoying bug that's spoilt a fair few movies.

Regression?

On 08/11/2010 09:00 PM, Rafael Gattringer wrote:
> Regression?
>

From a new Maverick Alpha 3 install & using '$ sudo
update-apt-xapian-index':

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+COMMAND
10244 root 20 0 124m 96m 10m R 95.2 26.1 1:26.92
update-apt-xapi

95.2% cpu - 26.1% mem.

Nice & IONICE are no help at all when running instances of ubuntu under a virtual machine (since the virtual machine itself doesn't get it's own priority lowered during that time).

ramayer, the weekly script checks for virtual environments and does not use ionice in this case. But the "apt" daily script also runs apt-xapian-update, for which I have filed bug 594820.

I can confirm this with up-to-date Maverick on HP NX6110 (Celeron M 1,4 GHz, 512 MB of RAM). It's eating 85 % of CPU and 20 % of MEM (according to top). This bug should be reopened.

Jiri, it is fair that it has a high CPU percentage if nothing else is using CPU. The original problem that it was making the system sluggish has been resolved, therefore this report should stay closed. Open new bugs for other issues.

The bug title is "update-apt-xapian-index uses too much CPU".

My system gets sluggish with it. This is the same report and therefore I think it would be foolish to spread it out to another bug.

Confirmed on Maverick, Lucid.

Did you read the bug description? Anyway, it is not productive to use an old bug report that has a multitude of different issues mentioned, and most of them are fixed. No developer can work efficiently with such a report any longer. It focused on the issue of the original bug reporter which has been fixed and released now. File your own bug report if it is still sluggish on your system. If only out of respect for all the people who are subscribed to this report.

I reported bug 655831. Please go over and confirm/me too it.

I have 10.10 with all update. And this process take ±98~100% CPU on a Atom n450 with 1g ram (Acer Aspire One),. that make other apps non responsive and very slow.

Also running an atom n4##. This process runs 98-100 cpu, and does so indefinitely. makes my fan noisy as the system heats up (which is why I found the process running, trying to find out why an idle system was running it's cpu at max for the past several minutes).

Manually replacing the apt-xapian-index package with the Maverick version solved this issue for me in Lucid. However I don't get why this fix isn't being officially released for Lucid. In comment #47 Tormod Volden suggested following instructions at https://wiki.ubuntu.com/StableReleaseUpdates, but this seems to be adressed to developers ("Upload the fixed package to release-proposed with the patch in the bug report, a detailed and user-readable changelog, and no other unrelated changes"), which I am not.

Can anybofy tell me why an existing fix to such a severe performance bug (on a netbook anyway) isn't being backported into the LTS, and what a user could do to get it there?

The upload itself has to be done by an approved developer, but anyone can help out with the documentation tasks and preparing the source package to be uploaded.

There is a second tweak to solve the problem, even in Ubuntu 10.10, suggested by Selanit in Ubuntu Forums (http://ohioloco.ubuntuforums.org/showpost.php?p=10453047&postcount=15)

Until now, the fix above applies only to weekly cron job. But the fix below will apply to the daily cron job.

open this file as sudo : /etc/cron.daily/apt

Find this code:
if [ -x /usr/sbin/update-apt-xapian-index ]; then
            nice ionice -c3 update-apt-xapian-index -q
        fi

Then replace it by that:
if [ -x /usr/sbin/update-apt-xapian-index ]; then
            nice -n 19 ionice -c3 update-apt-xapian-index --update -q
        fi

thank you "chtnh" for your observation!
this has to be fixed too.

ok, pulled, fixed and sent to apt maintainer.

can anyone of the bzr wizards confirm or deny that this is the right way to do it...?

bzr branch lp:apt
#change file and update changelog
bzr commit -m "fixed lp#363695 - apt-xapian-index should be nice"
#get mailaddress of apt maintainer
bzr send --body="hope this is the right way to submit fix for lp#363695" --<email address hidden>

How does one know the current status of this bug? I can see from aexl's previous comment "Fixed and sent to apt maintainer", but what does that mean status-wise?

This is more a curiosity question, seeing that I had to change my /etc/cron.daily/apt file this morning with the above change suggested by chtnh.

I'm running (grabbed from /etc/issue):
Ubuntu natty (development branch) \n \l

High CPU remains on an updated Natty. (Upgrade from Maverick)
Fragment in /etc/cron.daily/apt looks like this:
fi
 update_stamp $UPDATE_STAMP
 UPDATED=1
        # now run apt-xapian-index if it is installed to ensure the index
        # is up-to-date
        if [ -x /usr/sbin/update-apt-xapian-index ]; then
            nice ionice -c3 update-apt-xapian-index -q
        fi

Fix released ?

I installed and did all the upgrades Ubuntu Natty had.
But, update-apt-xapian-index when it launches uses 100% of my cpu.

I have an Intel Celeron 2.66 Ghz processor and this totally kills it, it's worse than with Vista, huge bug.

Reproducable on updated Natty. Please advise if a new bug should be reported and if you'd like to have a video with Conky.
Open software-center and aside run 'top' in terminal.
Almost immediately update-apt-xapi races CPU to ~100%.
Or search e.g. 'snort' brings up CPU almost to 100%.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12658 root 20 0 414m 233m 11m R 99 5.9 0:45.57 update-apt-xapi

Huge bug for me also. This is not fixed in Natty for sure.

(it is cool to post as "another_sam" behind "Sam_")

I have the same bug. I kill that process, but it appears again after some time. I haven't any Software Center or apt-get running, just Firefox, Kate, Transmission...

This is Ubuntu Natty on a HP Compaq laptop wih a Core 2 Duo. Should I/we report it as a new bug?

Same bug here. It's taking 100% of one of the the cores (HP Compaq laptop, Core 2 Duo around 3 years old, basic Natty). It happens some time after I kill it, over and over...

Here is an screenshot of htop http://i.imgur.com/HKzBD.png

Should I/we report it as a new bug?

Just happened to me. CPU up to 100%, and then my laptop temperature jumped to 96 degrees C, overheated, and shut itself off.

On 06/14/2011 05:13 PM, Robert Horswell wrote:
> Just happened to me. CPU up to 100%, and then my laptop temperature
> jumped to 96 degrees C, overheated, and shut itself off.
>

Reopen the bug. CPU overheating should is critical. Be sure to post your
details (Ubuntu version etc). You might use:

$ apport-collect 363695

to include all of the details.

Reproducable on Oneiric.
Open Software Center. CPU goes immediately up to 100%, after 10 sec. or so slows down again.

$ apport-collect 363695
ERROR: connecting to Launchpad failed: [Errno 1] Operation is not allowed: '/home/zootie/.launchpadlib'
You can reset the credentials by removing the file "/home/zootie/.cache/apport/launchpad.credentials"

I too have this bug on a brand new clean install of Natty on a laptop that it brings to a crawl. Terrible.

I too have this on natty - samsung nc10 - makes the computer nearly unusable!

I have the same problem in Natty. Intel Pentium 4 3GHz, 2GiB RAM
The service uses 99% CPU and the mouse doesn't respond.
Package atp-xapian-index 0.41ubuntu6
I have solve it: sudo aptitude purge apt-xapian-index

This is not yet fixed!

And it's what I would call a papercut. It makes Ubuntu feel unprofessional and not ready to be my only desktop OS.

Fix that helped me is in editing /usr/share/software-center/softwarecenter/backend/aptd.py:

- axi.update_async(True, False)
+ axi.update_async(False, False)

I can confirm this on MSI Wind U100 netbook with a 1.6 Ghz Intel Atom processor using Ubuntu 11.04 with 2.6.38-8 kernel.

Same problem here on Natty with AMD Sempron 2800+ and 1GB of RAM.

This bug seems quite old, and I'm surprised it hasn't been fixed yet!!

Never saw this problem in older versions like Edgy or Dapper though, I fail to see that this program is necessary for operation?

This still happens on 10.04. Can someone change the bug status? Or shall we open a new issue?

Confirmed on 11.04

Still happens on 11.10 server x86, apt-xapian-index 0.44ubuntu4
why the heck is this marked as fixed even though countless people say it isnt ?!

Still happens on 11.10 on 3.0.0-12-generic x86_64.

I guess one solution is to run

sudo rm /etc/cron.weekly/apt-xapian-index

Still happens on Kubuntu 11.10.

Same here in Oneiric 64bit. In half an hour it ran three times using nearly 100% CPU each time.

boozehead, if it was running several times, maybe you experienced bug #594820 ?

For those wondering why this bug report is marked Fixed, and who did not read the whole thread because it has become unreadable, see for instance comment 77. See also bug #655831 which is still open. As I explained there, 100% CPU usage in itself is not a problem.

okay guys. they're not going to look at the case or do anything if you keep adding to a bug that was marked as fixed.
you need to start a new bug.

> Date: Thu, 27 Oct 2011 06:59:08 +0000
> From: <email address hidden>
> To: <email address hidden>
> Subject: [Bug 363695] Re: update-apt-xapian-index uses too much CPU
>
> Same here in Oneiric 64bit. In half an hour it ran three times using
> nearly 100% CPU each time.
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (801007).
> https://bugs.launchpad.net/bugs/363695
>
> Title:
> update-apt-xapian-index uses too much CPU
>
> Status in APT:
> Confirmed
> Status in “apt-xapian-index” package in Ubuntu:
> Fix Released
>
> Bug description:
> Binary package hint: apt-xapian-index
>
> A background silent update process shouldn't use 98% CPU. It makes
> system sluggish with no visible reason.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/apt/+bug/363695/+subscriptions

http://sapnwnewbie.blogspot.com/2010/11/high-cpu-usage-by-update-apt-xapi.html

Found this. May help

same problem!!!!!!!!!!!!! ubuntu 10.10

i got the same problem, ubuntu 11.10, no way to solve it?

Same problem here - using Ubuntu 11.10, kernel 3.0.0-15-generic.
Uses 100% of one core.

Newer bug 903787
Workaround here for Oneiric and Precise, removed apt-xapian-index, no issues found yet.

I found something that helped my laptop to work properly ;) ... hope you find it useful too

1) http://ubuntuforums.org/showpost.php?p=9304431&postcount=8

2) http://ubuntuforums.org/showpost.php?p=10453047&postcount=15

In my opinion, this is not solved. With nice and ionice the system performance may be less affected, but the power drain by heavy CPU usage will still remain.

The question are: Does the update need to be that costly, and does the user need this updates altogether. As far as I see, the index provide two functions: The quick search mode in synaptic pagage manager, which is not installed by default, and the package recommenadtions in the terminal, rarely useful to normal users either, and expendable by power users if it comes at this large cost.
That's it, or have I forgotten something? If not, I would conclude that apt-xapian-index just should not be installed by default, and has to prevail for users that intentionally install it. It must be made sure however, that no one misses the features without knowledge how to get them back.

hello paul,

i don't know either, but i think opening and cross-linking a new issue "apt-xapian-index just should not be installed by default" will help get this clearer...!

This bug is currently affecting my old low-end PC running Ubuntu 12.04. The system become completely unusable when apt-xapian-index is running in the background.

This issue is far from solved. My machine is blocked for about 5 minutes regularly:

$ uname -a
Linux xander-pc 3.2.0-24-generic #37-Ubuntu SMP Wed Apr 25 08:43:22 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

$ sudo lshw -C cpu
  *-cpu
       description: CPU
       product: Intel(R) Celeron(R) CPU 2.93GHz
       vendor: Intel Corp.
       physical id: 4
       bus info: cpu@0
       version: Intel(R) Celeron(R) D CPU 2.93GHz
       serial: To Be Filled By O.E.M.
       slot: CPU 1
       size: 2933MHz
       capacity: 2933MHz
       width: 64 bits
       clock: 133MHz
       capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx x86-64 constant_tsc up pebs bts nopl pni dtes64 monitor ds_cpl tm2 cid cx16 xtpr lahf_lm

I forgot to mention that I do not have any RAM shortage (2 GB), so paging should not be the problem (top shows that there is enough free memory but update-apt-xapian-index uses about 11% of it).

this is pretty amazing you should give it a look http://www.inews15fl.net/biz/?read=4902269

Is comment 124 a spam?

I'm also still experiencing this problem, both on a high end machine (high end, but programs are using up all available memory usually), and a low end system.

Also have this problem with update-apt-xapian

Linux bimer 3.2.0-26-generic #41-Ubuntu SMP Thu Jun 14 16:26:01 UTC 2012 i686 i686 i386 GNU/Linux
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 12.04 LTS
Release: 12.04
Codename: precise

I may do some tests/checks if needed

Agreed - this is not fixed. Bug took one of my 12.04 servers out of commission. update-apt-xapi ate up 123MB of memory (the server only has 256MB) and then proceeded to top off the swap until the machine hung hard, forcing us to hard reboot.

Hello,

Please help, I can't get updates, it says update apt xapian index unfound

Affects me too, update-apt-xabian is using one core at 100% for a minute everyday.

Version:
apt-xapian-index 0.25ubuntu2

System:
Distributor ID: Ubuntu
Description: Ubuntu 10.04.4 LTS
Release: 10.04
Codename: lucid

Kernel:
2.6.32-45-generic x86_64

CPU:
Intel(R) Core(TM)2 Duo CPU U9400 @ 1.40GHz

If any more information are needed, I'd be happy to provide them.

The same problem on the old PC.
On modern PCs do not encounter this problem.

Lubuntu 12.10 - x86
Intel Celeron 1.7Ghz
768 RAM

update-apt-xapian-index
loads of 100% CPU

Help please!

Still a problem on 12.10 (64 bit) - all 4 CPU cores maxed out and hard disk being totally thrashed for several minutes every time I boot up and the desktop loads.

Bug is still there, Ubuntu 12.04.1 server 32bit, running on an old laptop. Process gets started but the daily apt cron job, and after a while it gets killed for running out of memory.

A weekly fatal unable to fork error message from /etc/cron.weekly/apt-xapian-index on a server with 512MB memory with a web server and database server running leads to this bug. That cron job calls /usr/sbin/update-apt-xapian-index, which is a Python script that imports axi and axi.indexer.

Sorting is a classic computer science problem with both time and space complexity. For time think CPU. For space think memory.

The solution is to change the sorting algorithm in the python axi and axi.indexer modules. The first priority is to switch to an algorithm that consumes a whole lot less memory (i.e., each step of the algorithm keeps less objects in memory), and it will stop crashing and stop thrashing (memory swapping to disk).

The second priority is of lesser importance (because renice can solve a lot of the effect), which is to switch to an algorithm that takes a lot less time to run (i.e., takes fewer steps to complete), and it will stop consuming so much CPU for so long.

On an idle system with 512M memory and 768M swap, while /etc/cron.weekly/apt-xapian-index was running, I used free to see memory and swap used. It rose to then peaked at this:

             total used free
Mem: 503528 497736 5792
Swap: 786428 231196 555232

Total used memory is 728M.

After ending, it immediately dropped back to this:

             total used free
Mem: 503528 259160 244368
Swap: 786428 224484 561944

Total used memory is 483M.

I conclude that apt-xapian-index consumes the difference, which is 245M.

Running "apt-cache stats" I see at the end "Total space accounted for: 26.0 M".

Therefore, it takes 245M to sort and index 26M of information. This seems conclusive that the algorithms, containers, and functions chosen are very inefficient. It should definitely not require 10 times the memory space of what is being indexed.

I added my above comments to bug 655831

I just stumbled on this bug today in raring. So, I confirm that the bug is still not fixed.

Yup, this exists in 13.04 all right.

I can confirm this bug is still affecting Ubuntu 12.04 LTS

Encountered this on a fresh instance of raring immediately after installing and updating apt-file. sudo apt-file purge immediately fixed the issue.

Bug encoutered in 14.04 (dev branch) also.

Please, fill new bug reports if you still find this issue. The usage is *by-design*. It was designed to not hog the CPU.

> "The usage is *by-design*. It was designed to not hog the CPU."
> "update-apt-xapian-index uses too much CPU"
I'm lost in these arguments.

Bralam's comment is irrelevant.

I just encountered the problem with LUbuntu 13.10 32-bit, after installing "Ubuntu Software Center". I then replicated on Ubuntu 13.10 32-bit.

System stats once "update-apt-xapian-index" begins: 99% CPU usage on FOUR CORES with 257MB of memory used by the process which renders the system completely unusable.

I'm seriously tempted to just re-write the entire program and push it all upstream for the sake of the community. This is RIDICULOUS! FIVE YEARS after the bug was noted and there is NO fix outside of simply *not using it*.

You're driving people away from Ubuntu by not fixing this.

My 12.04.5 LTS occasionally grinds to a halt with login nearly impossible. Finally I caught this remote embedded 256Mbyte system in the act and found update-apt-xapi had driven the machine well into swap. Hopefully apt-get remove apt-xapian-index will prevent this happening again until this long-standing defect is fixed. Perhaps mmap its big RAM data to /tmp/something and put in a substantial sleep in its main loop.

Reproduced on daily build of XUbuntu 15.04 kernel 3.19.0-12-generic

This happened to me today on XUbuntu 15.04 final with updates. It seems to stay on hogging as much as CPU as it can get its hands on (nice 10 apparently) and using about as much memory as stated earlier here. It just keeps on hogging, making me wonder if something's gone wrong.

I probably can figure out how to disable it somewhere..an average user would not.

Why not just limit this to 20% maximum of 1 core and 5% of memory? (14.04.2 LTS here). no one would even otice this any more.

The suggested fix and the currently applied fix differ in one important aspect: The --update option has been dropped.

tldr: Unless --update is broken (and it doesn't appear to be), it should be included in the fix as it reduces run time quite significantly.

To estimate the effect of --update I tried installing a number of small apps (e.g. htop) and running the command: update-apt-xapian-index --update (with default niceness). The time required to perform this update varied from about 40 seconds to almost 3 times that. The update process appears to be split into three main components:

- Reading translations for all ubuntu repos in use (security_universe, updates_restricted, main, universe, partner, multiverse, etc).
- Reading xapian index.
- Updating xapian index.

When a complete rebuild is performed, the index doesn't need to be read (and isn't).

In terms of time, the first and second steps are fairly constant, while the updating bit varies wildly with no pattern I was able to determine (from about 10 to 60 seconds).

Reading the index was quick, about 5 seconds.

The translations part, which I assume is basically fetching complete descriptions of all packages installed, is rather slow even on a fast internet connection. On my system, it took a consistent 25 seconds.

Without debating the merits of the index and its use, there are certainly a number of optimizations that could be made. Even without touching the process itself, just using the incremental update feature cuts run time significantly with no apparent downside.

This bug has been fixed upstream as of April 2010, but for some reason NOT in any of the Ubuntu packages since.

upstream:
http://anonscm.debian.org/cgit/collab-maint/apt-xapian-index.git/tree/debian/cron.weekly

ubuntu:
http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/wily/apt-xapian-index/wily/view/head:/debian/cron.weekly

This was done deliberately, so the bug should be modified to WONT FIX.

Agree or disagree, you can see the reason for the WONT FIX here:
http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/wily/apt-xapian-index/wily/revision/22

(One might argue that Software Center should handle the database being updated more gracefully. If anyone feels sufficiently strongly that way, I would recommend opening a bug on Software Center.)

* 2016-03-31 10:12 GMT+03:00 fyo wrote:
>
> Agree or disagree, you can see the reason for the WONT FIX here:
> http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/wily/apt-xapian-index/wily/revision/22

fyo, do you mean this statement:

> - do not modify the DB "in-place" with --update to avoid
> software-center seeing a corrupted database when it has
> it open at the same time

?

S.

> fyo, do you mean this statement:

> > - do not modify the DB "in-place" with --update to avoid
> > software-center seeing a corrupted database when it has
> > it open at the same time

Yes, that appears to be the argument for removing --update, which was otherwise added in an upstream patch.

It is unfortunate that update-apt-xapian-index resource use is increased many-fold due to Software Center not (gracefully) handling an in-progress update. Depending on the nature of the Software Center issue, this choice is perhaps understandable.

I should note that I was NOT able to reproduce any issue with Software Center while running update-apt-xapian-index --update. The closest thing I could get was a 10 second "animated loading ring" coinciding with the update process being at 99% (any io heavy activity could probably give that result, though).

* 2016-04-02 23:43 GMT+03:00 fyo wrote:
>
>> fyo, do you mean this statement:
>
>> > - do not modify the DB "in-place" with --update to avoid
>> > software-center seeing a corrupted database when it has
>> > it open at the same time
>
> Yes, that appears to be the argument for removing --update, which was
> otherwise added in an upstream patch.

I still fail to see how removing ionice binary (a disk I/O scheduler)
from the index rebuilding can possibly change DB modification type
from "in-place" to some other type:

--- quote ---
 then
         if [ -x "$IONICE" ]
         then
- nice -n 19 $IONICE -c 3 $CMD --update --quiet
+ nice -n 19 $IONICE -c 3 $CMD --quiet
         else
- nice -n 19 $CMD --update --quiet
+ nice -n 19 $CMD --quiet
         fi
 fi
--- quote ---

Have I missed something inbetween? This comment seems absolutely
irrelevant to me, to say the least. Maybe we need to contact Michael
Vogt to make the things clear?

> I should note that I was NOT able to reproduce any issue with Software
> Center while running update-apt-xapian-index --update. The closest
> thing I could get was a 10 second "animated loading ring" coinciding
> with the update process being at 99% (any io heavy activity could
> probably give that result, though).

That could prove the comment from Changelog is unrelated.

> It is unfortunate that update-apt-xapian-index resource use is increased
> many-fold due to Software Center not (gracefully) handling an in-
> progress update. Depending on the nature of the Software Center issue,
> this choice is perhaps understandable.

OK, I am not into design of this exact software solution, but if your
first statement is true then it's the design itself which is most
flawed here. IMHO, Software Centrer should notify user about the
update ongoing in the background. And then use a backup of the DB.
Until the new revision of DB is finalized. After which S.C. should
switch to the latter.

This is quick shot from a guy walking by.

S.

> I still fail to see how removing ionice binary (a disk I/O scheduler) from the index rebuilding can possibly change DB modification type

That's not what the diff says. There is no change in ionice behavior in this patch. The patch removes --update from both branches of the if-statement.

* 2016-04-03 23:40 GMT+03:00 fyo wrote:
>
> That's not what the diff says. There is no change in ionice behavior in
> this patch. The patch removes --update from both branches of the if-
> statement.

Thanks for pinpointing that to me, fyo -- somehow I have mixed these
four lines (and --update + --quiet options) up.

S.

This is still a problem in 16.04 LTS.

This process uses 100% CPU for a long time and up to 500MB of RAM on my machine. That's completely unacceptable.

Maybe now when Ubuntu switched to GNOME Software is a good time to add the --update option to both cron.weekly and cron.daily.

Does anybody actually know if this is a flaw of some settings, or the xapian engine by design? There are so many database and indexing systems out there, I can't get it why this single indexing job behaves so bad since years.

Confirming this issue with Ubuntu 14.04.3 LTS on an AWS t2.nano instance (only 512MB RAM).
The weekly cron job that is set up by default runs ~24h after machine creation and completely hogs the instance for more than 2 minutes.

IMHO: A process which is setup by default should not be so greedy.

Confirming this issue with Ubuntu 14.04.4 LTS on HP 630.
apt-xapian-index in /etc/cron.weekly uses 100% of my CPU.

Confirming that this is still a bug in Ubuntu 16.04.1 LTS (Xenial Xerus). Tested on a Pentium III Mobile 1GHz w/ 1GB ram. (Compaq Evo N600c).

Also, I note that this bug has been open for seven years and offer a workaround that solves the problem for me. Since the update-apt-xapian-index process is needlessly impolite even with nice and ionice, I wrote a script (called "stutter") that forces the process to pause repeatedly. (Technically, it uses SIGSTOP/SIGCONT with a 10% active duty cycle and 0.1s period).

It's a kludge, but it definitely works. As I'm posting this comment, I'm running update-apt-xapian-index in the background. Without my script, the streaming YouTube video I'm also playing in the background would have been unwatchably choppy instead of smooth.

To use my solution put the attached script into /usr/local/bin/stutter, make it executable, and update /etc/cron.weekly/apt-xapian-index to prepend stutter before the command:

==== /etc/cron.weekly/apt-xapian-index ====
#!/bin/sh

CMD=/usr/sbin/update-apt-xapian-index

# ionice should not be called in a virtual environment
# (similar to man-db cronjobs)
egrep -q '(envID|VxID):.*[1-9]' /proc/self/status || IONICE=/usr/bin/ionice

if [ -x "$IONICE" ]
then
    IONICE_ARGS="-c 3"
else
    IONICE=""
fi

# Check if we're on battery
if which on_ac_power >/dev/null 2>&1; then
    on_ac_power >/dev/null 2>&1
     ON_BATTERY=$?

    # Here we use "-eq 1" instead of "-ne 0" because
    # on_ac_power could also return 255, which means
    # it can't tell whether we are on AC or not. In
    # that case, run update-a-x-i nevertheless.
    [ "$ON_BATTERY" -eq 1 ] && exit 0
fi

# If we have B9's stutter utility installed, use it to
# prevent excessive interference with interactive users.
STUTTER=$(which stutter)

# Rebuild the index
if [ -x "$CMD" ]
then
    $STUTTER nice -n 19 $IONICE $IONICE_ARGS $CMD --quiet
fi

==== /usr/local/bin/stutter ====
#!/bin/bash

# stutter: Force a process to pause its work every once in a while.
# v1.0 (c) 2016 hackerb9 - GPLv3 or higher.
#
# Usage: stutter -p PID
# or: stutter command [arguments...]

# Even with the latest Linux (4.4, at the moment) it is possible for
# background processes to rudely prevent interactive use of a
# computer. This is true even with "nice -n 19 ionice -c 3 $CMD".
#
# [Yes, I'm looking at you, update-apt-xapian-index.]

# This kludge repeatedly sends STOP and CONT signals to a process to
# force it to give up the resource (CPU/disk/network) it is hogging.

# A reasonable default is to have the process active for 0.01s and
# sleep for 0.09s. (A 10% duty cycle with a period of 0.1s).
ontime=0.01
offtime=0.09

#verbose=true
debug() {
    if [ "$verbose" = "true" ]; then
 echo "$@" >&2
    fi
}

usage() {
    period=$(echo "$ontime + $offtime" | bc)
    duty=$(echo "$ontime * 100 / $period" | bc)

    cat <<EOF
stutter - Force a process to pause its work every once in a while
Usage: stutter [ -p <pid> | <cmd> ]

Current defaults
    on time: $ontime seconds
   off time: $offtime seconds
   (A $duty% duty cycle with a period of ${period}s).
EOF
}

cleanup() {
    if [ "$pid" ] && ps --pid $pid >/dev/null; then
 if [ "$processWasAlr...

Assuming there's no way that u-a-x-i can be re-written to avoid using so much ram, and so much CPU, the "stutter" kludge above makes it clear that it it ought to have sleeps in it. Use of nice (should be nice -19, in my book) and ionice have reduced the problem, but the
bug should still be open.

Still happily chewing CPU cycles on 16.10.

Another pretty effective workaround might be: apt-get remove --purge apt-xapian-index

The downside (on Kubuntu) it's that it takes out also muon, but if you don't depend on it, it makes a terrific difference on your laptop battery life.

Kubuntu 18.04, again the problem has not been solved, sometimes I find myself the laptop or desktop that slows down, looking through the processes there is always the process "apt-xapian-index" that uses CPU.
I resolve by removing the package, but it pulls other dependencies: kubuntu-driver-manager, kubuntu-notification-helper.
I know how to solve by removing the package, but a new user who does not know how to do it could blame Kubuntu or KDE.

Th bug was reported in 2009 and in Kubuntu 18.10 the bloody thing is still making my laptop unusable for several minutes every day after boot although it's not even in cron.daily. It is super annoying.

This bug was reported way back in 2009 . Today it's 1st June 2020 and this bug is still here kubuntu 18.04 . This background process fires up a few seconds after i turn on my laptop and continues to consume 100% of both the cores and after a minute or so when it stops . Python3 would then start consuming the CPU power , I suspect this again is something related to APT .

All of this is really annoying , during the time when this process is running you can't do anything .

I think this bug is Ubuntu-specific. It never happend since I'm using Devuan/Debian.

What is going on? Please make last 32-bit Ubuntu Kubuntu Lubuntu Xubuntu 18.04 usable out of box for everyone who will now come with old laptops to use 32-bit version.
Remove this bad package. Who is trolling and not allowing fix implemented?
I just installed old Laptop and on each start computer is unusable for few minutes since i dont need this, also uses electricity, cpu is slow, even if i put ssd, cause its doing a lot in slow cpu for long time few minutes.