Ubuntu
apt-xapian-index package

  1. Bug #363695
  2. Comment #135

Comment 135 for bug 363695

Revision history for this message
Max Polk (maxpolk) wrote : Re: update-apt-xapian-index uses too much CPU

A weekly fatal unable to fork error message from /etc/cron.weekly/apt-xapian-index on a server with 512MB memory with a web server and database server running leads to this bug. That cron job calls /usr/sbin/update-apt-xapian-index, which is a Python script that imports axi and axi.indexer.

Sorting is a classic computer science problem with both time and space complexity. For time think CPU. For space think memory.

The solution is to change the sorting algorithm in the python axi and axi.indexer modules. The first priority is to switch to an algorithm that consumes a whole lot less memory (i.e., each step of the algorithm keeps less objects in memory), and it will stop crashing and stop thrashing (memory swapping to disk).

The second priority is of lesser importance (because renice can solve a lot of the effect), which is to switch to an algorithm that takes a lot less time to run (i.e., takes fewer steps to complete), and it will stop consuming so much CPU for so long.