Ubuntu
puppet package

BIND (Ubuntu 10.04) DNS servers do not respond when forwarding for upstreams which return SERVFAIL

Bug #1020067 reported by ICT
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
puppet (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

Hi,

we have recently started installing Ubuntu 12.04 Server. We are using puppet and noticed some very odd behaviour on 12.04. When running puppet on 12.04 the run is very slow when querying our BIND DNS servers. The BIND DNS servers are setup as forwarders to our Windows 2008R2 Servers.

running tcpdump shows that puppet is trying to query AAAA records without the fullyqualified domain name. For some reason this has a huge impact, as the BIND servers don't respond to the client and so the clients runs in a timeout.

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
10:46:40.740415 IP 172.28.19.52.34257 > gedapvl01.a.space.corp.domain: 64350+ PTR? 0.0.0.0.in-addr.arpa. (38)
10:46:40.740699 IP 172.28.19.52.46768 > gedapvl01.a.space.corp.domain: 15589+ PTR? 11.16.28.172.in-addr.arpa. (43)
10:46:40.741133 IP 172.28.19.52.33600 > gedapvl01.a.space.corp.domain: 19407+ PTR? 52.19.28.172.in-addr.arpa. (43)
10:46:40.849382 IP 172.28.19.52.60961 > gedapvl01.a.space.corp.domain: 39593+ AAAA? puppet.a.space.corp. (37)
10:46:40.849870 IP 172.28.19.52.32963 > gedapvl01.a.space.corp.domain: 61770+ AAAA? puppet.a.space.corp.a.space.corp. (50)
10:46:40.851213 IP 172.28.19.52.43808 > gedapvl01.a.space.corp.domain: 53335+ A? puppet.a.space.corp. (37)
10:46:45.238025 IP 172.28.19.52.47328 > gedapvl01.a.space.corp.domain: 36971+ AAAA? puppet.a.space.corp. (37)
10:46:45.239324 IP 172.28.19.52.53076 > gedapvl01.a.space.corp.domain: 1888+ AAAA? puppet.a.space.corp.a.space.corp. (50)
10:46:45.240345 IP 172.28.19.52.53371 > gedapvl01.a.space.corp.domain: 34589+ A? puppet.a.space.corp. (37)
10:46:47.671963 IP 172.28.19.52.40283 > gedapvl01.a.space.corp.domain: 37664+ AAAA? puppet.a.space.corp. (37)
10:46:47.673165 IP 172.28.19.52.36495 > gedapvl01.a.space.corp.domain: 44009+ AAAA? puppet. (24)

As soon as it tries to resolve "AAAA? puppet." the puppet clients hangs. When the same machine is configured to query our Windows DNS servers instead of the Bind DNS Servers, the problem doesn't exist.

I'm not really sure if this problem is a bug but it only happens on Ubuntu 12.04. On Ubuntu 10.04 we don't see this behaviour.

The machines are configured by DHCP. I have started a thread regarding this issue on the ubuntu forums but no one seems to have a clue what is going wrong, so I was told to file a bug report.

Regards,
Oliver

Tags: bot-comment
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1020067/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

tags: added: bot-comment
ICT (oliver-weinmann-6)
affects: ubuntu → puppet (Ubuntu)
Revision history for this message
Robie Basak (racb) wrote : Re: Ubuntu 12.04 resolving hangs when querying AAAA records against BIND (Ubuntu 10.04) DNS servers

Thank you for taking the time to report this bug and helping to make Ubuntu better.

I have a bunch of questions come to mind when thinking about what could be causing this problem.

What is the exact tcpdump line you used? Is the BIND server really not sending any replies at all? If it is, please can you do another tcpdump and include them? If you're really not getting any responses, please could you use dig or host to generate a DNS query that does work so that we can see it in the tcpdump output to verify that it is working as expected?

Could you please attach /etc/resolv.conf from the machine running puppet?

You say that the problem goes away when querying your Windows DNS servers directly. Could you please post an equivalent tcpdump for comparison?

Marking as Incomplete pending answers to these questions. Please change the bug status back to New after you have responded. Thanks!

Changed in puppet (Ubuntu):
status: New → Incomplete
Revision history for this message
ICT (oliver-weinmann-6) wrote :
Download full text (6.4 KiB)

Hi,

** What is the exact tcpdump line you used? **

I run the following tcpdump client on the client:

**

I run nslookup gedaspw02 (gedaspw02 is a host on our local network) on the client and I do get a response:

root@ubuntu12043:/lhome/ict# nslookup gedaspw02
Server: 172.28.16.11
Address: 172.28.16.11#53

Non-authoritative answer:
Name: gedaspw02.a.space.corp
Address: 172.28.4.12

The corresponding tcpdump snippet:

root@ubuntu12043:/lhome/ict# tcpdump -i eth0 port 53
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
08:14:21.949355 IP 172.28.19.52.33205 > gedapvl01.a.space.corp.domain: 58103+ A? gedaspw02.a.space.corp. (40)
08:14:21.949683 IP 172.28.19.52.34745 > gedapvl01.a.space.corp.domain: 64598+ PTR? 11.16.28.172.in-addr.arpa. (43)
08:14:21.951179 IP gedapvl01.a.space.corp.domain > 172.28.19.52.33205: 58103 1/0/0 A 172.28.4.12 (56)
08:14:21.951191 IP gedapvl01.a.space.corp.domain > 172.28.19.52.34745: 64598 1/0/0 PTR gedapvl01.a.space.corp. (79)
08:14:21.951289 IP 172.28.19.52.53705 > gedapvl01.a.space.corp.domain: 10253+ PTR? 52.19.28.172.in-addr.arpa. (43)
08:14:21.952504 IP gedapvl01.a.space.corp.domain > 172.28.19.52.53705: 10253 NXDomain 0/1/0 (112)

When I start puppet, the startup is extremely slow, which lead me to the assumption that something with DNS is not working correctly. Here is the tcpdump snippet when running puppet:

08:17:08.497093 IP gedapvl01.a.space.corp.domain > 172.28.19.52.36834: 6015 2/0/0 A 91.189.95.55, A 91.189.95.54 (66)
08:17:09.775535 IP 172.28.19.52.56145 > gedapvl01.a.space.corp.domain: 44576+ AAAA? puppet.a.space.corp. (37)
08:17:09.776626 IP gedapvl01.a.space.corp.domain > 172.28.19.52.56145: 44576 0/1/0 (94)
08:17:09.776719 IP 172.28.19.52.43018 > gedapvl01.a.space.corp.domain: 28254+ AAAA? puppet. (24)
08:17:14.782915 IP 172.28.19.52.52767 > gedappl01.a.space.corp.domain: 28254+ AAAA? puppet. (24)
08:17:14.783060 IP 172.28.19.52.60943 > gedapvl01.a.space.corp.domain: 35223+ PTR? 13.16.28.172.in-addr.arpa. (43)
08:17:14.784074 IP gedapvl01.a.space.corp.domain > 172.28.19.52.60943: 35223 1/0/0 PTR gedappl01.a.space.corp. (79)
08:17:16.402411 IP 172.28.19.52.54017 > gedapvl01.a.space.corp.domain: 24380+ A? daisy.ubuntu.com. (34)
08:17:16.426397 IP gedapvl01.a.space.corp.domain > 172.28.19.52.54017: 24380 2/0/0 A 91.189.95.55, A 91.189.95.54 (66)
08:17:19.786837 IP 172.28.19.52.43018 > gedapvl01.a.space.corp.domain: 28254+ AAAA? puppet. (24)

It gets stuck when trying to resolve AAAA puppet.

** Could you please attach /etc/resolv.conf from the machine running puppet? **

root@ubuntu12043:/lhome/ict# cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 172.28.16.11
nameserver 172.28.16.13
search a.space.corp

The client is setup using DHCP.

** You say that the problem goes away when querying your Windows DNS servers directly. Could you please post an equivalent tcpdump for comparison? **

The same client using a windows DNS Server:

08:22:01.358496 IP gedas...

Read more...

Changed in puppet (Ubuntu):
status: Incomplete → New
Revision history for this message
Robie Basak (racb) wrote :

Oliver,

Thanks for the additional details.

Based on this I think it is clear that this is not a bug in puppet. Your Windows servers should not be responding SERVFAIL to AAAA queries. They should be responding with NOERROR and no answers. I think that bind is probably doing the right thing according to the specification here, although I'm not completely sure.

It seems to me that the behaviour that you're seeing is an unfortunate consequence of your Windows DNS servers not supporting IPv6 properly, by failing when they are queried for IPv6 addresses. Perhaps there is an option somewhere to turn it on, or an option in bind to work around broken upstream DNS servers?

Since my conclusion is that this is not a bug (neither in puppet nor bind) in Ubuntu, I'm marking this bug as invalid. However, I may be wrong. If you can point to an authoritative source that states that bind is supposed to respond differently when getting a SERVFAIL upstream, then please point to that and change the bug status back to New, and we'll retarget the bug at bind.

Changed in puppet (Ubuntu):
status: New → Invalid
summary: - Ubuntu 12.04 resolving hangs when querying AAAA records against BIND
- (Ubuntu 10.04) DNS servers
+ BIND (Ubuntu 10.04) DNS servers do not respond when forwarding for
+ upstreams which return SERVFAIL
Revision history for this message
Robie Basak (racb) wrote :

Just a thought: disabling IPv6 on the host running puppet may work - given that your DNS servers don't support IPv6 anyway. See https://wiki.ubuntu.com/IPv6#Disabling_IPv6 - although I don't see updated instructions for 12.04.

Revision history for this message
ICT (oliver-weinmann-6) wrote :

Problem solved by changing BIND configuration according to this article:

http://www.daemonforums.org/showthread.php?t=4471

Thanks.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.