network-manager stops and restarts already ifup'ed interfaces

Bug #90267 reported by James
58
This bug affects 2 people
Affects Status Importance Assigned to Milestone
network-manager (Ubuntu)
Fix Released
Critical
Alexander Sack
ntp (Ubuntu)
Invalid
Medium
Unassigned
upstart (Ubuntu)
Invalid
Medium
Unassigned

Bug Description

Ntp gets started before the network is setup. I have this happen with 2 machines. Each gets their network information via dhcp. One machine was installed from kubuntu herd4. The other was installed from kubuntu herd5. Both machines are kept up to date. The following happens consistently on both machines.

1. network manager starts
2. ntp is started but can only bind to lo because eth0 is not up yet
3. dhcp client brings up eth0
4. ntpdate is then run buts fails because ntp is already running

Mar 7 09:36:26 wilma NetworkManager: <information>^IWill activate wired connection 'eth0' because it now has a link.
Mar 7 09:36:26 wilma NetworkManager: <information>^ISWITCH: no current connection, found better connection 'eth0'.
Mar 7 09:36:26 wilma NetworkManager: <information>^IWill activate connection 'eth0'.
Mar 7 09:36:26 wilma NetworkManager: <information>^IDevice eth0 activation scheduled...
Mar 7 09:36:26 wilma NetworkManager: <information>^IActivation (eth0) started...
Mar 7 09:36:26 wilma NetworkManager: <information>^IActivation (eth0) Stage 1 of 5 (Device Prepare) scheduled...
Mar 7 09:36:26 wilma NetworkManager: <information>^IActivation (eth0) Stage 1 of 5 (Device Prepare) started...
Mar 7 09:36:26 wilma NetworkManager: <information>^IActivation (eth0) Stage 2 of 5 (Device Configure) scheduled...
Mar 7 09:36:26 wilma NetworkManager: <information>^IActivation (eth0) Stage 1 of 5 (Device Prepare) complete.
Mar 7 09:36:26 wilma NetworkManager: <information>^IActivation (eth0) Stage 2 of 5 (Device Configure) starting...
Mar 7 09:36:26 wilma NetworkManager: <information>^IActivation (eth0) Stage 2 of 5 (Device Configure) successful.
Mar 7 09:36:26 wilma NetworkManager: <information>^IActivation (eth0) Stage 3 of 5 (IP Configure Start) scheduled.
Mar 7 09:36:26 wilma NetworkManager: <information>^IActivation (eth0) Stage 2 of 5 (Device Configure) complete.
Mar 7 09:36:26 wilma NetworkManager: <information>^IActivation (eth0) Stage 3 of 5 (IP Configure Start) started...
Mar 7 09:36:27 wilma NetworkManager: <information>^IActivation (eth0) Beginning DHCP transaction.
Mar 7 09:36:27 wilma NetworkManager: <information>^IActivation (eth0) Stage 3 of 5 (IP Configure Start) complete.
Mar 7 09:36:27 wilma NetworkManager: <information>^IDHCP daemon state is now 12 (successfully started) for interface eth0
Mar 7 09:36:27 wilma rpc.statd[4877]: Version 1.0.10 Starting
Mar 7 09:36:27 wilma ntpd[4896]: ntpd 4.2.2p4@1.1585-o Tue Nov 28 12:19:49 UTC 2006 (1)
Mar 7 09:36:27 wilma ntpd[4897]: precision = 1.000 usec
Mar 7 09:36:27 wilma ntpd[4897]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Mar 7 09:36:27 wilma ntpd[4897]: Listening on interface wildcard, ::#123 Disabled
Mar 7 09:36:27 wilma ntpd[4897]: Listening on interface lo, ::1#123 Enabled
Mar 7 09:36:27 wilma ntpd[4897]: Listening on interface lo, 127.0.0.1#123 Enabled
Mar 7 09:36:27 wilma ntpd[4897]: kernel time sync status 0040
Mar 7 09:36:27 wilma hcid[4914]: Bluetooth HCI daemon
Mar 7 09:36:27 wilma ntpd[4897]: frequency initialized -6.663 PPM from /var/lib/ntp/ntp.drift
Mar 7 09:36:28 wilma NetworkManager: <debug info>^I[1173224188.060423] nm_hal_device_added (): New device added (hal udi is '/org/freedesktop/Hal/devic
es/platform_bluetooth').
Mar 7 09:36:28 wilma hcid[4914]: Starting SDP server
Mar 7 09:36:28 wilma NetworkManager: <information>^IDHCP daemon state is now 1 (starting) for interface eth0
Mar 7 09:36:28 wilma dhclient: DHCPREQUEST on eth0 to 255.255.255.255 port 67
Mar 7 09:36:28 wilma dhclient: DHCPACK from 172.16.31.1
Mar 7 09:36:28 wilma avahi-daemon[4723]: Joining mDNS multicast group on interface eth0.IPv4 with address 172.16.31.3.
Mar 7 09:36:28 wilma avahi-daemon[4723]: New relevant interface eth0.IPv4 for mDNS.
Mar 7 09:36:28 wilma avahi-daemon[4723]: Registering new address record for 172.16.31.3 on eth0.IPv4.
Mar 7 09:36:28 wilma avahi-daemon[4723]: Withdrawing address record for 172.16.31.3 on eth0.
Mar 7 09:36:28 wilma avahi-daemon[4723]: Leaving mDNS multicast group on interface eth0.IPv4 with address 172.16.31.3.
Mar 7 09:36:28 wilma avahi-daemon[4723]: Interface eth0.IPv4 no longer relevant for mDNS.
Mar 7 09:36:28 wilma avahi-daemon[4723]: Joining mDNS multicast group on interface eth0.IPv4 with address 172.16.31.3.
Mar 7 09:36:28 wilma avahi-daemon[4723]: New relevant interface eth0.IPv4 for mDNS.
Mar 7 09:36:28 wilma avahi-daemon[4723]: Registering new address record for 172.16.31.3 on eth0.IPv4.
Mar 7 09:36:28 wilma NetworkManager: <information>^IDHCP daemon state is now 4 (reboot) for interface eth0
Mar 7 09:36:28 wilma NetworkManager: <information>^IActivation (eth0) Stage 4 of 5 (IP Configure Get) scheduled...
Mar 7 09:36:28 wilma NetworkManager: <information>^IActivation (eth0) Stage 4 of 5 (IP Configure Get) started...
Mar 7 09:36:28 wilma NetworkManager: <information>^IRetrieved the following IP4 configuration from the DHCP daemon:
Mar 7 09:36:28 wilma NetworkManager: <information>^I address 172.16.31.3
Mar 7 09:36:28 wilma NetworkManager: <information>^I netmask 255.255.255.0
Mar 7 09:36:28 wilma NetworkManager: <information>^I broadcast 172.16.31.255
Mar 7 09:36:28 wilma NetworkManager: <information>^I gateway 172.16.31.254
Mar 7 09:36:28 wilma NetworkManager: <information>^I nameserver 172.16.31.1
Mar 7 09:36:28 wilma NetworkManager: <information>^I domain name 'bedrock'
Mar 7 09:36:28 wilma NetworkManager: <information>^IActivation (eth0) Stage 5 of 5 (IP Configure Commit) scheduled...
Mar 7 09:36:28 wilma NetworkManager: <information>^IActivation (eth0) Stage 4 of 5 (IP Configure Get) complete.
Mar 7 09:36:28 wilma NetworkManager: <information>^IActivation (eth0) Stage 5 of 5 (IP Configure Commit) started...
Mar 7 09:36:28 wilma avahi-daemon[4723]: Withdrawing address record for 172.16.31.3 on eth0.
Mar 7 09:36:28 wilma avahi-daemon[4723]: Leaving mDNS multicast group on interface eth0.IPv4 with address 172.16.31.3.
Mar 7 09:36:28 wilma avahi-daemon[4723]: Interface eth0.IPv4 no longer relevant for mDNS.
Mar 7 09:36:28 wilma avahi-daemon[4723]: Joining mDNS multicast group on interface eth0.IPv4 with address 172.16.31.3.
Mar 7 09:36:28 wilma avahi-daemon[4723]: New relevant interface eth0.IPv4 for mDNS.
Mar 7 09:36:28 wilma avahi-daemon[4723]: Registering new address record for 172.16.31.3 on eth0.IPv4.
Mar 7 09:36:28 wilma dhclient: bound to 172.16.31.3 -- renewal in 20870 seconds.
Mar 7 09:36:29 wilma NetworkManager: <information>^IClearing nscd hosts cache.
Mar 7 09:36:29 wilma NetworkManager: <WARNING>^I nm_spawn_process (): nm_spawn_process('/usr/sbin/nscd -i hosts'): could not spawn process. (Failed to
execute child process "/usr/sbin/nscd" (No such file or directory))
Mar 7 09:36:29 wilma NetworkManager: <information>^IActivation (eth0) successful, device activated.
Mar 7 09:36:29 wilma NetworkManager: <information>^IActivation (eth0) Finish handler scheduled.
Mar 7 09:36:29 wilma NetworkManager: <information>^IActivation (eth0) Stage 5 of 5 (IP Configure Commit) complete.
Mar 7 09:36:31 wilma avahi-daemon[4723]: Registering new address record for fe80::203:ceff:fe88:fb2e on eth0.*.
Mar 7 09:36:31 wilma ntpdate[5143]: the NTP socket is in use, exiting

Revision history for this message
Mark Reitblatt (mark-reitblatt) wrote :

I believe this is actually a problem w/ the program that starts up ntp.

Changed in ntp:
importance: Undecided → Medium
Changed in upstart:
importance: Undecided → Medium
Revision history for this message
C de-Avillez (hggdh2) wrote :

Confirmed. But this is not a ntp or upstart bug.

If you are running off wireless, then only when you log in will gnome-network-manager (or knetworkmanager if under KDE) be run -- and this is when, eventually, ntpdate gets kicked.

NTP is started during the boot process (I think as a S50) if it is installed.

So... in fact, n-m should not run ntpdate if ntpd is running. Marking as a n-m bug.

Changed in network-manager:
status: Unconfirmed → Confirmed
C de-Avillez (hggdh2)
Changed in network-manager:
assignee: nobody → desktop-bugs
C de-Avillez (hggdh2)
Changed in upstart:
status: Unconfirmed → Rejected
Changed in ntp:
status: Unconfirmed → Rejected
Revision history for this message
James (boddingt) wrote :

No wireless here. As a work around I put a sleep 5 at the top of /etc/init.d/ntp. This delays the starting of ntp long enough that I now have a working ntp.

Revision history for this message
C de-Avillez (hggdh2) wrote :

This is interesting. NTP should, by default, bind to all local interfaces, on UDP, and should be able to recover by itself.

Can you attach a 'grep ntp /var/log/daemon.log' piece here, long enough to show the system startup & some few hours of run?

Changed in network-manager:
importance: Undecided → Medium
Revision history for this message
James (boddingt) wrote :

Ntp seems to go out of it's way to not do a wildcard bind to port 123. It is only binding to port 123 for each interface. If the interface is not up it does not get bound to. This is from starting ntp after eth0 is up.

Mar 9 06:36:46 fred ntpd[2276]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Mar 9 06:36:46 fred ntpd[2276]: Listening on interface lo, 127.0.0.1#123 Enabled
Mar 9 06:36:46 fred ntpd[2276]: Listening on interface eth0, 172.16.31.2#123 Enabled

I started another machine and let it go for 30 minutes and no difference. Ntp does not change.

Mar 9 07:52:34 wilma ntpd[5034]: ntpd 4.2.2p4@1.1585-o Wed Mar 7 20:43:30 UTC 2007 (1)
Mar 9 07:52:34 wilma ntpd[5035]: precision = 1.000 usec
Mar 9 07:52:34 wilma ntpd[5035]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Mar 9 07:52:34 wilma ntpd[5035]: Listening on interface wildcard, ::#123 Disabled
Mar 9 07:52:34 wilma ntpd[5035]: Listening on interface lo, ::1#123 Enabled
Mar 9 07:52:34 wilma ntpd[5035]: Listening on interface lo, 127.0.0.1#123 Enabled
Mar 9 07:52:34 wilma ntpd[5035]: kernel time sync status 0040
Mar 9 07:52:34 wilma ntpd[5035]: frequency initialized -8.102 PPM from /var/lib/ntp/ntp.drift

By 8:22 no change.

Revision history for this message
James (boddingt) wrote :

I just tried ntp 4.2.4 which has code to scan for new interface every 5 minutes. That will bind to eth0 sometime after eth0 is brought up.

There is another problem with ntp starting before eth0 is up. When the server lines in ntp.conf use hostnames they can not be resolved. So if ntp does eventually bind to eth0 it still won't work.

Revision history for this message
C de-Avillez (hggdh2) wrote :

I will look more closely at what is going on, James. On your last comment: yes, this is a known, huh, issue/limitation: if there is no name server resolution available, then fully-qualified hostnames (like ntp.ubuntu.com, or pool.ntp.org, etc) will not get resolved -- after all, they do depend on having access to a NS. Usually, when synchronisation is critical, NTP should be set with an IP address, not a FQHN.

But, once bound to a working interface, NTP should eventually be able to access a NS, and synchronise.

Right now it is starting to look like we have two different issues here:

(1) when NTP is installed, network-manager should not call ntpdate;

(2) NTP cannot bind to an interface that is not available during startup (this one is a bit more convoluted, methinks: if n-m is the one setting up any, and all network interfaces, then it seems NTP should not be started until n-m successfully brings up such an interface).

Do you agree?

Revision history for this message
C de-Avillez (hggdh2) wrote :

and... yes indeed. Per chance I had started my laptop with a wireless connection, driven by n-m.

Here's the NTP messages for the startup:

Mar 8 18:28:00 localhost ntpd[8566]: ntpd 4.2.2p4@1.1585-o Wed Mar 7 20:43:39 UTC 2007 (1)
Mar 8 18:28:00 localhost ntpd[8567]: precision = 1.000 usec
Mar 8 18:28:00 localhost ntpd[8567]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Mar 8 18:28:00 localhost ntpd[8567]: Listening on interface wildcard, ::#123 Disabled
Mar 8 18:28:00 localhost ntpd[8567]: Listening on interface lo, ::1#123 Enabled
Mar 8 18:28:00 localhost ntpd[8567]: Listening on interface lo, 127.0.0.1#123 Enabled
Mar 8 18:28:00 localhost ntpd[8567]: kernel time sync status 0040
Mar 8 18:28:00 localhost ntpd[8567]: frequency initialized 12.637 PPM from /var/lib/ntp/ntp.drift
Mar 8 18:28:00 localhost ntpd[8575]: signal_no_reset: signal 17 had flags 4000000
Mar 8 18:28:02 localhost ntpd_initres[8575]: host name not found: ntp.ubuntu.com
Mar 8 18:28:02 localhost ntpd_initres[8575]: couldn't resolve `ntp.ubuntu.com', giving up on it
Mar 8 18:28:02 localhost ntpd_initres[8575]: host name not found: pool.ntp.org
Mar 8 18:28:02 localhost ntpd_initres[8575]: couldn't resolve `pool.ntp.org', giving up on it
Mar 8 18:33:55 localhost ntpdate[9707]: the NTP socket is in use, exiting
Mar 8 20:33:33 localhost ntpd[8567]: ntpd exiting on signal 15

So we have the interfaces disabled, everywhere. NTP is up, but is just wasting CPU cycles and memory. The errors on name resolution are expected, since I do not have a caching NS running on my laptop. Would not help much, anyways :-)

So I restarted it, now with the wireless running:

Mar 8 20:33:36 localhost ntpd[14447]: ntpd 4.2.2p4@1.1585-o Wed Mar 7 20:43:39 UTC 2007 (1)
Mar 8 20:33:36 localhost ntpd[14448]: precision = 1.000 usec
Mar 8 20:33:36 localhost ntpd[14448]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Mar 8 20:33:36 localhost ntpd[14448]: Listening on interface wildcard, ::#123 Disabled
Mar 8 20:33:36 localhost ntpd[14448]: Listening on interface vmnet8, fe80::250:56ff:fec0:8#123 Enabled
Mar 8 20:33:36 localhost ntpd[14448]: Listening on interface lo, ::1#123 Enabled
Mar 8 20:33:36 localhost ntpd[14448]: Listening on interface eth1, fe80::290:4bff:feff:c508#123 Enabled
Mar 8 20:33:36 localhost ntpd[14448]: Listening on interface lo, 127.0.0.1#123 Enabled
Mar 8 20:33:36 localhost ntpd[14448]: Listening on interface eth1, 192.168.1.3#123 Enabled
Mar 8 20:33:36 localhost ntpd[14448]: Listening on interface vmnet8, 192.168.54.1#123 Enabled
Mar 8 20:33:36 localhost ntpd[14448]: kernel time sync status 0040
Mar 8 20:33:36 localhost ntpd[14448]: frequency initialized 12.637 PPM from /var/lib/ntp/ntp.drift

and my wireless interface, eth1, is now being used by NTP.

Hum. I will have to munch a bit on this. Your comments will be appreciated.

Revision history for this message
Matt Zimmerman (mdz) wrote :

ntp ought to be restarted when a new interface is brought up, but for some reason this is disabled by default:

mizar:[/etc/network/if-up.d] cat ntp
#!/bin/sh

# remove (or comment out) the next line if your network addresses change
exit 0
case $ADDRFAM in
        inet*)
                if [ -x "/etc/init.d/ntp" ]; then
                        if [ -x "`which invoke-rc.d 2>/dev/null`" ]; then
                                invoke-rc.d ntp restart || exit $?
                        else
                                /etc/init.d/ntp restart || exit $?
                        fi
                fi
                ;;
esac

Revision history for this message
James (boddingt) wrote :

Hggdh,
I agree with your points.

I tried commenting out the exit 0 in the script Matt mentions. Ntp works at boot time now. It also gets restarted if I ifdown eth0; ifup eth0. Still have ntpdate being run after ntp has started and it fails because ntp is running. In ntp.conf a mix of server lines using IP and FQDN works fine.

Watching the boot process it looks like ntp is being started early on then being restarted after the network is up.

Revision history for this message
C de-Avillez (hggdh2) wrote :

more digging in required (thanks, Matt, your comments, and of those in #ubuntu-bugs helped a lot).

I am always amazed by the depth of my ignorance...

/etc/network/if-(up|down|post-down|pre-up).d scripts are the responsibility of each package: NTP, NTPdate, and others. These scripts are executed whenever the action (reflected in the directory name) is executed.

So, it is not a n-m issue here. I was wrong. Nor it is NTP (not sure), or upstart. Will have to figure out.

Anyway: a summary, so far:

1. as provided, ubuntu has the /etc/network/if-up.d/ntp script disabled. As a result, connections purely managed from n-m do not get ntp service (so it seems). Why?

2. Although not hurting, ntp and ntpdate got installed; ntpdate cannot run if ntpd is running, so having both in *may* not be necessary.

Revision history for this message
C de-Avillez (hggdh2) wrote :

There is a comment in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=399905 about this issue, and it refers to an upstream bug (https://ntp.isc.org/bugs/show_bug.cgi?id=51) where it is strongly recommended not to bounce ntpd on I/F status changes.

This is (probably) the reason why /etc/network/ifup.d/ntp is disabled by default: there is a chance the clock will get wildly inaccurate, and the "carefully maintained statistics" will be lost.

Per the comments on https://ntp.isc.org/bugs/show_bug.cgi?id=772 upstream it seems that ntp, once it gives up on root, cannot change the IP (which is correct behaviour, since port 123 is privileged). The fix is either to run NTP as root, or to run Linux with capabilities (so that the correct capability can be given to the 'ntp' userid). Anyway, the fix is on 4.2.4, which has not yet been packaged by Debian.

So it seems there is no real solution right now.

Perhaps one option is not to install ntpd on machines that dynamically and/or frequently change the IP address -- instead, use ntpdate. On the other hand, ntpdate is less reliable (see 'man ntpdate' for example), and it is recommended to be run every hour in order to minimise the chance of large clock changes.

Still digging.

Changed in network-manager:
status: Confirmed → Needs Info
Revision history for this message
James (boddingt) wrote :

That comment is a good argument for leaving 'exit 0' in /etc/network/if-up/ntp when the move is made to ntp 4.2.4. I have already been playing with ntp 4.2.4 and seen the interface scanning in action. I think this is a solution for a different problem not the problem I am having.

What you said covers having a properly running ntp then having an interface change. My problem is ntp not starting up properly.

Not part of this bug but ntp 4.2.4 has it's own problems starting before the network is up when the computer is booting.

1. Starts before eth0 is up. Will refuse to add servers even by ip because there is no network to the address.
2. Can end up starting at the same time as ntpdate. If ntpdate gets port 123 first then ntp will fail to bind to port 123 and exit.
3. Have seen ntp start with eth0 up just in time so some servers can get added by ip. Not all servers listed by ip get added.
4. Still can not resolve any fqdn.

The computer that I have been pasting the log from has an ip of 172.16.31.3. The ip does not change. The computer gets it's ip via dhcp but is effectively a static ip as the ip is mapped to the mac address. My stratum 1 is 172.16.31.201.

ntp 4.2.2p4
When run before eth0 is up ntp can not bind to eth0. The server 172.16.31.201 can be added by ip but not by fqdn. 172.16.31.3 can send packets to 172.16.31.201 but because ntp is not bound to eth0 any replies will be ignored.

ntp 4.2.4
When run before eth0 is up ntp can not bind to eth0. Ntp refuses to add server 172.16.31.201 because eth0:172.16.31.3 is not up yet. Says "Cannot find existing interface for address 172.16.31.201". This version of ntp seems to pick up eth0 when it does it's scan ok but this does not do much good because no servers are defined.

If I restart ntp after the machines have booted then ntp will work properly everytime. Either version.

Revision history for this message
C de-Avillez (hggdh2) wrote :

James, you are correct. I had initially rejected this bug for NTP and upstart, but I am starting to wonder.

The more I read... the more it seems that current (i.e., 4.2.0) NTP code should not be started before a valid network interface is available.

There is a comment in the NTP Bugzilla that some work was/is being done on that, but I have not yet confirmed if it is already available or still dev. I am guessing, right now, it is not available (otherwise you would not have the issue on 4.2.2 and 4.2.4 :-) The bug in question, https://ntp.isc.org/bugs/show_bug.cgi?id=51, shows in the bug lists for versions up to, and including 4.2.4p0 (so, again, it sounds like it was either not added in 4.2.4, or did not work as expected -- huh, as I would expect).

There are two different views here, AFAIU: laptops & machines that do not have a constant network presence, and servers that are continuously connected. For the latter, NTP startup, as it is now, is OK.

For the former... it does not seem like this is kosher. And here the plot thickens: there are network-manager supported machines, and there are (as of now) non-network-manager supported ones.

Right now I am tending to state NTP should not be started if there are no network interfaces running; but I do not know what should be done when an interface changes its address (which, I know, is not your problem).

Marking as confirmed for NTP, leaving as confirmed for network-manager.

Changed in ntp:
status: Rejected → Confirmed
Changed in network-manager:
status: Needs Info → Confirmed
Revision history for this message
C de-Avillez (hggdh2) wrote :

I have asked for opinions from the assignee for bug # 82335 (it might need to be considered for the n-m work there).

Revision history for this message
James (boddingt) wrote :

Between that other thread and your comments I was hoping network manager might be optional. I removed it with apt-get remove network-manager

Getting sane behaviour. Network services being started after the network is up. Ntp working as expected. Servers specified by IP being added. Servers specified by name being added.

Every thing was working fine. Did an apt-get upgrade. Whoops, something in the upgrade pulled network-manager back in and back to the broken behaviour. I am guessing it was kubuntu-desktop. It was a freshly installed machine so a big update.

Removing n-m now brings it up to 4 work arounds I have just to get ntp working.

Revision history for this message
C de-Avillez (hggdh2) wrote :

James, with the changes that Tollef made for n-m, do you still experience this problem?

Revision history for this message
James (boddingt) wrote :
Download full text (3.3 KiB)

Still experiencing the problem. The following still shows ntp starting before the network is up when using n-m.

Mar 16 10:54:46 wilma NetworkManager: <information>^IDHCP daemon state is now 1 (starting) for interface eth0
Mar 16 10:54:48 wilma rpc.statd[4932]: Version 1.0.10 Starting
Mar 16 10:54:49 wilma ntpd[4951]: ntpd 4.2.2p4@1.1585-o Wed Mar 7 20:43:30 UTC 2007 (1)
Mar 16 10:54:49 wilma ntpd[4952]: precision = 1.000 usec
Mar 16 10:54:49 wilma ntpd[4952]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Mar 16 10:54:49 wilma ntpd[4952]: Listening on interface wildcard, ::#123 Disabled
Mar 16 10:54:49 wilma ntpd[4952]: Listening on interface lo, ::1#123 Enabled
Mar 16 10:54:49 wilma ntpd[4952]: Listening on interface lo, 127.0.0.1#123 Enabled
Mar 16 10:54:49 wilma ntpd[4952]: kernel time sync status 0040
Mar 16 10:54:49 wilma hcid[4969]: Bluetooth HCI daemon
Mar 16 10:54:49 wilma NetworkManager: <debug info>^I[1174006489.245454] nm_hal_device_added (): New device add
ed (hal udi is '/org/freedesktop/Hal/devices/platform_bluetooth').
Mar 16 10:54:49 wilma hcid[4969]: Starting SDP server
Mar 16 10:54:49 wilma ntpd[4952]: frequency initialized -8.412 PPM from /var/lib/ntp/ntp.drift
Mar 16 10:54:50 wilma dhclient: DHCPREQUEST on eth0 to 255.255.255.255 port 67
Mar 16 10:54:50 wilma dhclient: DHCPACK from 172.16.31.1
Mar 16 10:54:50 wilma ntpd[4952]: sendto(172.16.31.201) (fd=16): Network is unreachable
Mar 16 10:54:50 wilma avahi-daemon[4657]: Joining mDNS multicast group on interface eth0.IPv4 with address 172
.16.31.3.
Mar 16 10:54:50 wilma avahi-daemon[4657]: New relevant interface eth0.IPv4 for mDNS.
Mar 16 10:54:50 wilma avahi-daemon[4657]: Registering new address record for 172.16.31.3 on eth0.IPv4.
Mar 16 10:54:50 wilma avahi-daemon[4657]: Withdrawing address record for 172.16.31.3 on eth0.
Mar 16 10:54:50 wilma avahi-daemon[4657]: Leaving mDNS multicast group on interface eth0.IPv4 with address 172
.16.31.3.
Mar 16 10:54:50 wilma avahi-daemon[4657]: Interface eth0.IPv4 no longer relevant for mDNS.
Mar 16 10:54:50 wilma avahi-daemon[4657]: Joining mDNS multicast group on interface eth0.IPv4 with address 172
.16.31.3.
Mar 16 10:54:50 wilma avahi-daemon[4657]: New relevant interface eth0.IPv4 for mDNS.
Mar 16 10:54:50 wilma avahi-daemon[4657]: Registering new address record for 172.16.31.3 on eth0.IPv4.
Mar 16 10:54:50 wilma NetworkManager: <information>^IDHCP daemon state is now 4 (reboot) for interface eth0
Mar 16 10:54:50 wilma NetworkManager: <information>^IActivation (eth0) Stage 4 of 5 (IP Configure Get) schedul
ed...
Mar 16 10:54:50 wilma NetworkManager: <information>^IActivation (eth0) Stage 4 of 5 (IP Configure Get) started
...
Mar 16 10:54:50 wilma NetworkManager: <information>^IRetrieved the following IP4 configuration from the DHCP d
aemon:
Mar 16 10:54:50 wilma NetworkManager: <information>^I address 172.16.31.3
Mar 16 10:54:50 wilma NetworkManager: <information>^I netmask 255.255.255.0
Mar 16 10:54:50 wilma NetworkManager: <information>^I broadcast 172.16.31.255
Mar 16 10:54:50 wilma NetworkManager: <information>^I gateway 172.16.31.254
Mar 16 10:54:50 wilma NetworkManager: <information>^I nameserver 17...

Read more...

Revision history for this message
James (boddingt) wrote :

My prefer fix is still apt-get remove network-manager

I just installed network-manager 0.6.4-6ubuntu4 and it still starts ntpd before the network is up. I apt-get remove network-manager and ntp works properly again. With n-m not installed the network gets started THEN any network services are run.

These days I apt-get remove network-manager on any new install to avoid this problem.

Revision history for this message
C de-Avillez (hggdh2) wrote :

@James: I have just connected to my home wireless (on reboot) and... NTP came in correctly, after the wireless was connected! I am not sure what to attribute this to yet, but I can state I am running the stock feisty, up-to-date.

Would you be able to try it? I do not know about wired interfaces (I have one, but I will have to grab the "emergency" cable, which is very well and safely stored at an (right now) unknown location -- in other words, I lost it).

This sort of surprised me, since I have seen no developer activity here.

Revision history for this message
James (boddingt) wrote :

I just tried n-m 0.6.4-ubuntu5. Same problem.

Same reaction, an immediate apt-get remove network-manager to restore working behaviour.

Changed in network-manager:
assignee: desktop-bugs → nobody
Revision history for this message
Martin Pitt (pitti) wrote :

The n-m side of the problem is that it tears down and re-dhcdbd's interfaces which are already ifup'ed by /etc/init.d/networking. This definitively needs to be fixed, but is not release-critical.

The ntpd side of the problem is that it should be configured to just listen on * instead of per-interface. Or, alternatively, uninstalling n-m is an appropriate solution, too. :)

Revision history for this message
James (boddingt) wrote :

The ntp devs refuse to do a normal wild card bind and stick with what they have done. This has come up in comp.protocols.time.ntp quite a few times. Which I think led to the new interface scanning code in ntp 4.2.4 which has it's own problems when ntp is started before the network is up. I mentioned this in my 2007-03-10 post.

Starting ntp before the network is up also breaks name resolution for any servers that are specified by name. How can a server name be resolved to ip if the network is not up yet? The following is valid

server ntp.ubuntu.com
server 0.au.pool.ntp.org
server 1.au.pool.ntp.org
server 2.au.pool.ntp.org

but wil fail because the dns can not be reached because the network is not up.

These problems disappear if ntp is not started until after the network is up.

Revision history for this message
C de-Avillez (hggdh2) wrote :

There are some comments, in the ntp mailing lists, that on dynamic network environments, the following sequence should be used:

1. bring up the network
2. if this is boot time, run ntpdate as soon as possible during boot.
3. run ntpd -qg every so often (like every hour or so).

But I certainly could not see any comments there, nor see any option in 4.2.2, to make ntpd listen on * (and re-scan on a potential IP address change).

Revision history for this message
James (boddingt) wrote :

I would not use those steps. The 2 other solutions I remember from the comp.protocols.time.ntp archive which allow you to run ntp with a dynamic ip are

1. Use /etc/ppp/if-up.local to restart ntp each time ppp0 got a new ip. Does lose many hours of history but still keeps ntp working and the machine can also be used to serve time to other computers on the network. Small price to pay. Lose many hours history but keep a working ntp or have ntp stop working when ppp0 got a new ip was not a hard decision to make.

2. Have another machine on the network running 24x7 acting as a ntp server. The downside is yet another computer running all the time.

For a couple of years I went with option 1. It worked and gave me a ntp server for everything else. Option 2 happened when I decided to make my own stratum 1. The interface scanning in the newer ntp solve the problem created by option 1.

Ntpdate and ntpd -qg do the same thing. Just do it slightly differently. May as well pick one and and stick with it. One thing about using ntpd -g is it will step if the offset is > 128ms so you can get away with not using ntpdate at all.

To get ntp 4.2.2 to do a normal wild card bind you get to modify ntpd/ntp_io.c and rebuild the package. The rescanning bit is in a more recent version of ntp than what ubuntu uses.

Revision history for this message
C de-Avillez (hggdh2) wrote :

James, I am just presenting what I have found. I myself have my own stratum 1 at home, but this still does not resolve the issue between n-m and ntpd. In fact, for me it is even worse, since my laptop fluctuates betwen hotel, cellular-based, and home LANs. Using n-m, for me, is easier that hand-adapting to every different hotel I get to go.

Right now I am giving up & having ntpd restart every time a connection is made, and everytime the local IP changes. As you said, not a very difficult decision.

Revision history for this message
Martin Pitt (pitti) wrote :

Let's fix this for gutsy.

Revision history for this message
Martin Pitt (pitti) wrote :

Confirmed at UDS that this is a high-priority bug fix.

Right now, n-m assumes that all interfaces are down and uninitialized at startup. It needs to detect the state of interfaces and initialize the internal model accordingly. This should happen by looking at ifconfig/iwconfig to remain upstream compatible (no ifupdown specifics).

Martin Pitt (pitti)
Changed in ntp:
status: Confirmed → Rejected
Changed in network-manager:
importance: Medium → Critical
Revision history for this message
Joey Stanford (joey) wrote :

This is a "me too" post. It's causing "issues" on my two wireless laptops but is not an issue on my static ip'd and cabled desktop. Thanks for looking into this.

Revision history for this message
Neil Woolford (neil-neilwoolford) wrote :

Also a "me to". Exactly the same as Joey Stanford. My (one) wireless laptop with 7.04 and Network-Manager needs 'sudo /etc/intit.d/ntp restart' to get ntp running.

The cabled workstation, also with 7.04 and Network-Manager works fine after normal booting.

Happy to test fixes, provide logs etc if this is of help.

Neil

Revision history for this message
Neil Woolford (neil-neilwoolford) wrote :

Result of /var/log/daemon.log | grep ntp | less, last seven lines are after a restart of ntpd.

My wireless connection is on eth1.

Jun 19 10:19:47 neilw-laptop ntpd[5345]: ntpd 4.2.2p4@1.1585-o Wed Mar 7 20:43:30 UTC 2007 (1)
Jun 19 10:19:47 neilw-laptop ntpd[5346]: precision = 1.000 usec
Jun 19 10:19:47 neilw-laptop ntpd[5346]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Jun 19 10:19:47 neilw-laptop ntpd[5346]: Listening on interface wildcard, ::#123 Disabled
Jun 19 10:19:47 neilw-laptop ntpd[5346]: Listening on interface lo, ::1#123 Enabled
Jun 19 10:19:47 neilw-laptop ntpd[5346]: Listening on interface lo, 127.0.0.1#123 Enabled
Jun 19 10:19:47 neilw-laptop ntpd[5346]: kernel time sync status 0040
Jun 19 10:19:47 neilw-laptop ntpd[5346]: frequency initialized 8.088 PPM from /var/lib/ntp/ntp.drift
Jun 19 10:19:49 neilw-laptop ntpd_initres[5359]: host name not found: ntp2a.mcc.ac.uk
Jun 19 10:19:49 neilw-laptop ntpd_initres[5359]: couldn't resolve `ntp2a.mcc.ac.uk', giving up on it
Jun 19 10:19:49 neilw-laptop ntpd_initres[5359]: host name not found: ntp.cs.strath.ac.uk
Jun 19 10:19:49 neilw-laptop ntpd_initres[5359]: couldn't resolve `ntp.cs.strath.ac.uk', giving up on it
Jun 19 10:21:01 neilw-laptop ntpdate[5933]: the NTP socket is in use, exiting
Jun 19 10:21:03 neilw-laptop ntpdate[5602]: can't find host ntp2a.mcc.ac.uk
Jun 19 10:21:03 neilw-laptop ntpdate[5602]: the NTP socket is in use, exiting
Jun 19 10:21:15 neilw-laptop ntpdate[5823]: can't find host ntp2a.mcc.ac.uk
Jun 19 10:21:15 neilw-laptop ntpdate[5823]: the NTP socket is in use, exiting
Jun 19 10:23:39 neilw-laptop ntpd[5346]: ntpd exiting on signal 15
Jun 19 10:23:41 neilw-laptop ntpd[6111]: ntpd 4.2.2p4@1.1585-o Wed Mar 7 20:43:30 UTC 2007 (1)
Jun 19 10:23:41 neilw-laptop ntpd[6112]: precision = 1.000 usec
Jun 19 10:23:41 neilw-laptop ntpd[6112]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Jun 19 10:23:41 neilw-laptop ntpd[6112]: Listening on interface wildcard, ::#123 Disabled
Jun 19 10:23:41 neilw-laptop ntpd[6112]: Listening on interface eth1, fe80::213:2ff:feb8:af04#123 Enabled
Jun 19 10:23:41 neilw-laptop ntpd[6112]: Listening on interface lo, ::1#123 Enabled
Jun 19 10:23:41 neilw-laptop ntpd[6112]: Listening on interface lo, 127.0.0.1#123 Enabled
Jun 19 10:23:41 neilw-laptop ntpd[6112]: Listening on interface eth0:avahi, 169.254.7.105#123 Enabled
Jun 19 10:23:41 neilw-laptop ntpd[6112]: Listening on interface eth1, 192.168.1.101#123 Enabled
Jun 19 10:23:41 neilw-laptop ntpd[6112]: kernel time sync status 0040
Jun 19 10:23:41 neilw-laptop ntpd[6112]: frequency initialized 8.088 PPM from /var/lib/ntp/ntp.drift

Martin Pitt (pitti)
Changed in network-manager:
assignee: nobody → asac
Revision history for this message
Alexander Sack (asac) wrote :

Hi,

can you please post a complete daemon.log (or syslog) .... I need a complete log starting with the line where NetworkManager is initially started up.

Thanks,

 - Alexander

Revision history for this message
Neil Woolford (neil-neilwoolford) wrote :

Today's full daemon.log as requested.

I've not trimmed it at all, just in case...

The problem is still present on the system, with NTP not running properly ("No association IDs returned")

At 19:34:10 I manually restarted ntp, just for completenes. It works fine, as you can see at the end of the log.

Neil

Revision history for this message
Alexander Sack (asac) wrote :

network-manager (0.6.5-0ubuntu7) gutsy; urgency=low

  [Alexander Sack <email address hidden>]
  * prefetch bugfix from svn - not-connected wired interface is if-upped:
    1. 24a_svn2578-gnome354565-fix-ethernet-link-detection-races.patch:
      - prefetch patch from 0.6.0 release branch (rev 2578)
    2. 24b_svn2605-gnome354565-dont-up-notwired-interfaces.patch:
      - prefetch patch from 0.6.0 release branch (rev 2605)
  * debian/patches/25_lp90267-dont-tear-down-upped-interfaces.patch: fix master bug that
    makes already configured interfaces (in /etc/network/interfaces) being torn down. This
    causes issue for network cards that don't have a link beat and might caus troubles
    for applications that have already bound to the interface and don't react on netlink
    events (LP: #90267)

  [Anthony Mercatante <email address hidden>]
  * fixed network-manager.install and debian/rules:
    - Installs nm-vpn-perperties in /usr/lib to make it a hidden binary
    - Tell dh_shlibdeps to ignore nm-vpn-properties to avoid lots of gnome
      dependancies, causing issues to release kubuntu Kubuntu.

 -- Alexander Sack <email address hidden> Thu, 12 Jul 2007 11:15:56 +0200

Changed in network-manager:
status: Confirmed → Fix Released
Revision history for this message
folkoy (pluron) wrote :

The problem is fixed for me in Gutsy Tribe 5. My wired ethernet interface is up when I log in the GUI. Before I had to use the network manager applet and pick "wired interface" for it to come up.

Revision history for this message
Neil Woolford (neil-neilwoolford) wrote :

I'm afraid that the problem has persisted for me right through to using Gutsy updated to release candidate plus today's updates.

That's network-manager 0.6.5-0ubuntu15

Revision history for this message
Neil Woolford (neil-neilwoolford) wrote :

Attaching daemon.log for 15th October. Complete with successful restart of ntp at tail end.

Revision history for this message
Caleb Callaway (enlightened-despot) wrote :

"Me too" on this situation. Clean installation of the latest Gutsy, with all updates applied.

My fix has been to disable the ntp daemon startup in the ntp scripts, and have the daemon started through an if-up.d script, which simply sends a 'start' signal to /etc/init.d/ntp. This is workable, although I haven't tested how it will work if link goes down. If I understand things correctly, the if-up.d script should fail semi-gracefully when the link comes back up, and ntp should be able to engage in synchronization activities again.

From what I can tell, installing the NTP service (instead of using ntpdate, which is the default) actually *disables* time sync via NTP in Gutsy, because of the race conditions created.

Part of the problem is the NTP server's behavior, but it seems to be mostly a problem with how Network Manager communicates link status to other services. I may be wrong, but it seems that services started from the /etc/rc*.d/ directories have no way of knowing the status of links that are brought up via Network Manager, outside of checking a flag set by an if-up.d script (this strikes me as a bit of a hack).

Revision history for this message
Attila Darázs (darazs) wrote :

The bug is still not solved!

I have similar experience with an up-to-date Gutsy system, installed about 2 weeks ago.

So first ntp can't start, because /etc/network/if-up.d/ntpdate runs in the same time. If I disable the ntpdate script (write an "exit 0" in the beginning) then ntp starts, but _before_ my network card is up (I'm using DHCP), so ntp can't resolve the server names , can't find any server and it doesn't keep the time.

So please *Open this bug* again and look into it. I don't want to remove the network-manager, as others suggested, because it removes ubuntu-desktop.

Thanks,
   Attila

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.