dhclient not honoring prepend config

posted Nov 2, 2015, 5:52 AM by Tyler Akins   [ updated Nov 2, 2015, 5:54 AM ]
I have a CentOS 6.7 image running in AWS.  It reads /etc/dhcp/dhclinet.conf - here is mine:

timeout 300;
retry 60;
prepend domain-name-servers 127.0.0.1;
prepend domain-search "node.consul";

Please note that I fixed the first two lines!  The stock version of this file did not have semicolons at the end of the timeout nor retry lines.

The idea is that I would prefer to use the local dnsmasq before falling back to other domain name servers.  Sounds like a typical use case, right?  I can use "dhclient -r ; dhclient" to release and renew the DHCP lease and I see the entry in my /etc/resolv.conf exactly as I would expect.

nameserver 127.0.0.1
# other nameservers listed later in the file.

I believe that this works just fine and so I go ahead and reboot the box.  Just to be sure, I double check the /etc/resolv.conf file and find out, to my horror ... the line is missing!

Where did it go?  It totally worked before!  Running "dhclient -r ; dhclient" again puts the line back.  What's the deal?

It turns out that the version of dhclient that's installed (version 4.1.1 49.P1.el6.centos) is not properly setting $new_domain_name_servers for the REBOOT reason.  On reboot, dhclient will talk DHCP to a server and get a new lease.  It fires off /sbin/dhclient-script with $reason set to REBOOT.  When I use "dhclient -r ; dhclient" it does almost the same thing but $reason is set to BOUND.

The strange thing is that the environment variables are different for those two calls.  For REBOOT the $new_domain_name_servers does not list 127.0.0.1 and for BOUND it does list 127.0.0.1.  It should always have 127.0.0.1 because we have the "prepend domain-name-servers 127.0.0.1" config set.

I tried taking a peek at the source code but did not invest enough time to determine the cause for this issue.  I mostly gave up for these reasons:
  1. It wasn't obvious when I browsed the source code.  Investigating would take hours of work and I would likely have to add many debug lines and trace the execution manually.
  2. I would have to update dhclient.  This change wouldn't really roll out to older machines, which is exactly where I need this fix.
  3. Newer machines are switching away from dhclient.  Instead, they use NetworkManager or other alternatives.
  4. There's a quick and easy workaround to make this act as expected.
Let's talk about that workaround.  On CentOS, the /sbin/dhclient-script script is made to be extended.  It looks for /etc/dhcp/dhclient-enter-hooks and will execute it if it exists and is flagged as executable.  You can modify $new_domain_name_servers directly here.  So, just omit the "prepend domain-name-servers 127.0.0.1" in /etc/dhcp/dhclient.conf and instead you should create /etc/dhcp/dhclient-enter-hooks with the content here.

#!/bin/sh
# Prepend 127.0.0.1 to the list of name servers if new servers are being set.
if [ -n "$new_domain_name_servers" ]; then
    new_domain_name_servers="127.0.0.1 $new_domain_name_servers"
fi

A simple "chmod 0755 /etc/dhcp/dhclient-enter-hooks" and you're done.  This will always prepend 127.0.0.1 to your list of domain name servers.  The same method can work for all sorts of properties that dhclient is having difficulty honoring.

Problem solved.

It is possible that this isn't a "Problem Only Tyler Has".  Here's a few people that could be the result of the same issue as me or possibly a related issue.  They didn't solve it the same way I did and I didn't investigate their problems further to determine if they were really experiencing the same issue as myself.
Comments