cron stopped executing jobs

Asked by Nick Demou

I'm running a backup script from cron for many months
I found no backups for the past 20 days so I checked the logs
Here is what I see today (June the 21st):

> grep /var/log/cron.daily *
 | tail -n 3
syslog.0:May 31 07:30:01 lion anacron[24122]: Will run job `cron.daily' in 5 min.

syslog.0:May 31 07:35:01 lion anacron[24122]: Job `cron.daily' started

syslog.0:May 31 07:35:01 lion anacron[24312]: Updated timestamp for job `cron.daily' to 2011-05-31

What happened? What should I do in order to never happen again?

I must note that the last thing the backup script does is to execute an "apt-get upgrade -y"
Jobs stopped the last time apt-get upgrade run and updated libpam-runtime,libpam0g, libpam-modules,libisccc30 [see output of tail -n 29 /var/log/apt/term.log bellow]
Is there any connection?

It's very depressing when such things happen :(

________________________________
output of tail -n 29 /var/log/apt/term.log
Log started: 2011-05-31 07:47:32
(Reading database ... 109654 files and directories currently installed.)
Preparing to replace libpam-runtime 0.99.7.1-5ubuntu6.1 (using .../libpam-runtime_0.99.7.1-5ubuntu6.3_all.deb) ...
Unpacking replacement libpam-runtime ...
Setting up libpam-runtime (0.99.7.1-5ubuntu6.3) ...

(Reading database ... 109654 files and directories currently installed.)
Preparing to replace libpam0g 0.99.7.1-5ubuntu6.1 (using .../libpam0g_0.99.7.1-5ubuntu6.3_i386.deb) ...
Unpacking replacement libpam0g ...
Setting up libpam0g (0.99.7.1-5ubuntu6.3) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline

Processing triggers for libc6 ...
ldconfig deferred processing now taking place
(Reading database ... 109654 files and directories currently installed.)
Preparing to replace libpam-modules 0.99.7.1-5ubuntu6.1 (using .../libpam-modules_0.99.7.1-5ubuntu6.3_i386.deb) ...
Unpacking replacement libpam-modules ...
Setting up libpam-modules (0.99.7.1-5ubuntu6.3) ...

(Reading database ... 109654 files and directories currently installed.)
Preparing to replace libisccc30 1:9.4.2.dfsg.P2-2ubuntu0.6 (using .../libisccc30_1%3a9.4.2.dfsg.P2-2ubuntu0.7_i386.deb) ...
Unpacking replacement libisccc30 ...
Setting up libisccc30 (1:9.4.2.dfsg.P2-2ubuntu0.7) ...

Processing triggers for libc6 ...
ldconfig deferred processing now taking place
Log ended: 2011-05-31 07:47:41

Question information

Language:
English Edit question
Status:
Solved
For:
Ubuntu cron Edit question
Assignee:
No assignee Edit question
Solved by:
Michael Basse
Solved:
Last query:
Last reply:
Revision history for this message
Nick Demou (ndemou) said :
#1

Digging a little more I found thousands of errors like the following in auth.log starting at May 31 07:48 (no such error previously):

May 31 07:48:01 lion CRON[25367]: PAM unable to dlopen(/lib/security/pam_env.so)
May 31 07:48:02 lion CRON[25367]: PAM [error: /lib/libpam.so.0: version `LIBPAM_MODUTIL_1.1.3' not found (required by /lib/security/pam_
env.so)]
May 31 07:48:02 lion CRON[25367]: PAM adding faulty module: /lib/security/pam_env.so
May 31 07:48:02 lion CRON[25367]: pam_unix(cron:session): session opened for user root by (uid=0)

Now it really looks like the upgrade of libpam was the reason that cron stopped executing jobs.

The question remaining open is: What should I do in order to never happen again? Is there a better way to upgrade the system than apt-get -y upgrade?

Revision history for this message
Best Michael Basse (michael-alpha-unix) said :
#2

in my opinion its never a good idea to upgrade a system silenty and answering everything with yes. instead you can use "apticron" whoch is checking if there are updates and if so, you will get a mail about the updates with CVE and so on and then YOU have to do the update by HAND using "sudo apt-get dist-upgrade"

Revision history for this message
Nick Demou (ndemou) said :
#3

Thanks for the tip Michael, I didn't know about apticron.

> in my opinion its never a good idea to upgrade a system silently and answering everything with yes

I understand your point but I finally got tired from the constant stream of updates and decided to give the Wrong-Way a try. To be honest one of the reasons of going the wrong-way was that in cases like this I don't believe that sitting at the console and doing the upgrade manually, will make any difference: I would never have guessed that updating libpam would stop cron and would have hit yes anyway. Maybe there would have been some warning but during the last several years the only warnings I've seen during apt-get upgrade are either buried between huge amounts of low-priority messages or are the ones about restarting after a kernel update.

Revision history for this message
Nick Demou (ndemou) said :
#4

Thanks Michael Basse, that solved my question.