UniFi Security Gateway & Cloudflare's DNS over HTTPS

Infrastructure:

Introduction

The goal of this  post is to document how to succesfully configure a Unifi USG to run dnscrypt-proxy configured to use DNS over HTTPS with Cloudflare's 1.1.1.1 DNS resolver and to implement some firewall rules to limit unencrypted DNS resolution from leaving your home network. If succesfully, the end result will allow most devices to automatically have all of their DNS queries encrypted via HTTPS to Cloudflare automatically.

Standards

It's important to note that there are two primary RFC's that implement encrypted DNS traffic. In this tutorial, we'll be setting up DNS over HTTPS. However, there is also DNS over TLS. Both RFC's are linked below:

RFC 8484 - DNS Queries over HTTPS (DoH)
DNS Queries over HTTPS (DoH) (RFC 8484, October 2018)
RFC 7858 - Specification for DNS over Transport Layer Security (TLS)
Specification for DNS over Transport Layer Security (TLS) (RFC 7858, May 2016)

Cloudflare has also published an article that covers the differences here: https://www.cloudflare.com/learning/dns/dns-over-tls/. I also reccomend checking out https://blog.cloudflare.com/announcing-the-results-of-the-1-1-1-1-public-dns-resolver-privacy-examination/ for further privacy details around the Cloudflare 1.1.1.1 service.

Prerequisites

Before we start setting up things, take a moment to review your current network configuration. For this post, we're going to assume that you haven't attempted to install dnscrypt-proxy before on your USG and have no left over "cruft" that might get in the way. You should be able to SSH into your USG and Unifi CloudKey. Take note of any VLAN, VPN, or firewall configurations that might get in the way. This post makes no guarantees, please proceed with caution.

Installation

The dnscrypt-proxy project is hosted publically on Github here: https://github.com/DNSCrypt/dnscrypt-proxy . At the time of this writting, each release automatically is built for numerous CPU architectures but, most importantly, MIPS64. You can review this at https://github.com/DNSCrypt/dnscrypt-proxy/releases/latest/  and search for dnscrypt-proxy-linux_mips64.

  1. Download the latest MIPS64 release of dnscrypt-proxy to /tmp on your Unifi USG and extract it.
ssh admin@USG.IP.Address
sudo su - 
cd /tmp

export DNSCRYPT_VERSION="2.0.45"
curl --location --progress --remote-name "https://github.com/DNSCrypt/dnscrypt-proxy/releases/download/$DNSCRYPT_VERSION/dnscrypt-proxy-linux_mips64-$DNSCRYPT_VERSION.tar.gz"
tar xvfz "./dnscrypt-proxy-linux_mips64-$DNSCRYPT_VERSION.tar.gz"

2. Verify the downloaded version:

root@MainRouter:/tmp# ./linux-mips64/dnscrypt-proxy -version
2.0.44

4. Create a persistant directory and move the dnscrypt-proxy binary into place:

cd /config
mkdir dnscrypt-proxy
mv /tmp/linux-mips64/dnscrypt-proxy /config/dnscrypt-proxy/

5. Install the dnscrypt-proxy SysVInit script and create the initial configuration file:

root@MainRouter:~# /config/dnscrypt-proxy/dnscrypt-proxy -service install
[2020-04-11 17:09:54] [NOTICE] Installed as a service. Use `-service start` to start

vi /config/dnscrypt-proxy/dnscrypt-proxy.toml

6. Create /config/dnscrypt-proxy/dnscrypt-proxy.toml with the following.

# Only use cloudflare's IPv4 and IPv6 DNS services
server_names = ['cloudflare', 'cloudflare-ipv6']

# Listen on eth1's IPv4 address and loopback for IPv4 & IPv6.
# Default: 127.0.0.1:53
listen_addresses = ['172.16.1.1:6878', '127.0.0.1:6878', '[::1]:6878']

# Allow all possible internal IP addresses to establish a connection.
# Adjust for your network size.
# Default: 250
max_clients = 1024

# Use both IPv4 and IPv6 servers
# Default: true
ipv4_servers = true
# Default: false
ipv6_servers = true

# After binding to listen_addresses, drop to the nobody user
# Default: Continue running as same user
user_name = "nobody"

# Only use DoH servers, do not use dnscrypt servers
# Default: true
doh_servers = true
# Default: false
dnscrypt_servers = false

# Require servers to support DNSSec, not save logs, and not filter.
# Notably, this only matters if we add additional server_names above, Cloudflare supports all of these.
# Default: false
require_dnssec = true
# Default: false
require_nolog = true
# Default: false
require_nofilter = true

# If DoH fails, fall back to regular DNS via Cloudflare
# Default: 9.9.9.9:53
fallback_resolvers = ['1.1.1.1:53', '1.0.0.1:53']

# Do not fall back to the system DNS configuration
# Notably, the USG doesn't have this.
# Default: false
ignore_system_dns = true

# Send a SYN packet to Cloudflare to determine if basic internet connectivity can be established.
# Default: 9.9.9.9:53
netprobe_address = '1.1.1.1:53'

# Enable DNS caching to improve subsequent queries for our home network
cache = true
# Defaults to 512, 4x the cache size
cache_size = 4096
# Default: 60
cache_min_ttl = 2400
# Default: 86400
cache_max_ttl = 86400

block_undelegated = true

[blocked_names]
  blocked_names_file = 'blocklist.txt'
  log_file       = '/dev/stdout'

[schedules]
  [schedules.'work']
      mon = [{after='10:00', before='17:00'}]
      tue = [{after='10:00', before='17:00'}]
      wed = [{after='10:00', before='17:00'}]
      thu = [{after='10:00', before='17:00'}]
      fri = [{after='10:00', before='17:00'}]
  [schedules.'freetime']
      mon = [{after='17:00', before='23:00'}]
      tue = [{after='17:00', before='23:00'}]
      wed = [{after='17:00', before='23:00'}]
      thu = [{after='17:00', before='23:00'}]
      fri = [{after='17:00', before='23:00'}]
  [schedules.'sleep']
      mon = [{after='00:00', before='08:00'}]
      tue = [{after='00:00', before='08:00'}]
      wed = [{after='00:00', before='08:00'}]
      thu = [{after='00:00', before='08:00'}]
      fri = [{after='00:00', before='08:00'}]

# Enable the public dnscrypt-proxy sources list from which we configure cloudflare
# Notably, this list includes hundreds of DNS servers you can use.

[sources]
  [sources.'public-resolvers']
    urls = ['https://raw.githubusercontent.com/DNSCrypt/dnscrypt-resolvers/master/v3/public-resolvers.md', 'https://download.dnscrypt.info/resolvers-list/v3/public-resolvers.md']
    cache_file = '/tmp/public-resolvers.md'
    minisign_key = 'RWQf6LRCGA9i53mlYecO4IzT51TGPpvWucNSCh1CBM0QTaLn73Y7GFO3'
    prefix = ''

6. Start dnscrypt-proxy

service dnscrypt-proxy start

USG Configuration

At the time of this writting, the Unifi CloudKey WebUI does not expose the ability to forward DNS queries to a specific port.

Notably, the USG software ships with dnsmasq and additionally, as of the time of this writing, does not provide a user configuration point to disable the DNS resolver on port :53. The result is that we need to have dnscrypt-proxy listen on 6878 and configure dnsmasq to forward DNS queries to that port.

This creates a minor bump as the Unifi WebUI also does not support setting a custom port to forward DNS traffic to. It only supports a custom IPv4 address configuration even though dnsmasq will happily support forwarding to a customer IP address and port.

To make this work, we will leverage the https://help.ubnt.com/hc/en-us/articles/215458888-UniFi-USG-Advanced-Configuration option for the USG.

  1. SSH to your Unifi CloudKey
  2. vim /srv/unifi/data/sites/default/config.gateway.json
{
  "service":{
    "dns":{
      "forwarding":{
        "options":[
          "ptr-record=1.1.16.172.in-addr.arpa,MainRouter",
          "all-servers",
          "cname=unifi.dathouse,unifi",
          "server=127.0.0.1#6878"
        ]
      }
    }
  }
}

 3. Make sure that the unifi user has access to the configuration file by runninng the following:  chown unifi:unifi /srv/unifi/data/sites/default/config.gateway.json && chmod 0644 /srv/unifi/data/sites/default/config.gateway.json

4. Log into the Unifi WebUI and navigate to: Devices --> Select your USG --> Click on Config --> Click on Advanced --> Click the Provision buttonn. This will force the USG to be provisioned which will deploy your custom config.gateway.json along with all of the settings you've setup in the WebUI.

5. Back on the USG, you can use sudo mca-ctrl -t dump-cfg | less to verify that the dns forwarding config matches what you've setup in your config.gateway.json file.

Firewall based DNS redirect

At this point, all DNS traffic that is set to your USG should be redirected by dnsmasq through to dnscrypt-proxy which will then use DNS over HTTPS to resolve the request only via CloudFlare.

However, this setup requires all devices to not have local DNS settings and leverage the DHCP advertised DNS server address. There's a variety of ways to proceed and handle these but, the most straight forward will be to configure your USG to masquerade traffic that is leaving your home network on TCP port 53, to your local dnscrypt-proxy service on port 6878.

If you'd like to verify the unencrypted DNS traffic, you can leverage tcpdump on your USG to look for traffic on you WAN interface, eth0 using port 53. The following command does that and will decode and show the raw query. tcpdump -i eth0 dst port 53 or src port 53 -n -x -X -v

Example:

root@MainRouter:~# tcpdump -i eth0 dst port 53 or src port 53 -n -x -X -v
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
17:30:04.303255 IP (tos 0x0, ttl 63, id 39290, offset 0, flags [none], proto UDP (17), length 69)
    1.2.3.4.61619 > 8.8.8.8.53: 47743+ [1au] A? techsmix.net. (41)
	0x0000:  4500 0045 997a 0000 3f11 7b84 443e 125c  E..E.z..?.{.D>.\
	0x0010:  0808 0808 f0b3 0035 0031 2528 ba7f 0120  .......5.1%(....
	0x0020:  0001 0000 0000 0001 0874 6563 6873 6d69  .........techsmi
	0x0030:  7803 6e65 7400 0001 0001 0000 2910 0000  x.net.......)...
	0x0040:  0000 0000 00                             .....
17:30:04.385291 IP (tos 0x20, ttl 122, id 29232, offset 0, flags [none], proto UDP (17), length 85)
    8.8.8.8.53 > 1.2.3.4.61619: 47743$ 1/0/1 techsmix.net. A 192.241.192.75 (57)
	0x0000:  4520 0055 7230 0000 7a11 679e 0808 0808  E..Ur0..z.g.....
	0x0010:  443e 125c 0035 f0b3 0041 6219 ba7f 81a0  D>.\.5...Ab.....
	0x0020:  0001 0001 0000 0001 0874 6563 6873 6d69  .........techsmi
	0x0030:  7803 6e65 7400 0001 0001 c00c 0001 0001  x.net...........
	0x0040:  0000 012b 0004 c0f1 c04b 0000 2902 0000  ...+.....K..)...
	0x0050:  0000 0000 00                             .....

The above example shows an unencrypted DNS query passing through to the Google DNS servers performed manually with dig @8.8.8.8 techsmix.net from a device within my local network. This query is completly unencrypted and can be easily logged by any device on the network path between my home internet connection and Google.

That all aside, we once again hit a limitation of the Unifi WebUI. The WebUI does not allow configuring NAT firewall rules, so we need to add more configuration to our /srv/unifi/data/sites/default/config.gateway.json file.

  1. Update your config.gateway.json to look like the following
{
  "service":{
    "nat":{
      "rule":{
        "1":{
          "description":"dnscrypt-proxy DNS redirect",
          "destination":{
            "address":"!172.16.1.1",
            "port":"53"
          },
          "inbound-interface":"eth1",
          "inside-address":{
            "address":"172.16.1.1",
            "port":"6878"
          },
          "protocol":"tcp_udp",
          "type":"destination"
        }
      }
    },
    "dns":{
      "forwarding":{
        "options":[
          "ptr-record=1.1.16.172.in-addr.arpa,MainRouter",
          "all-servers",
          "cname=unifi.dathouse,unifi",
          "server=127.0.0.1#6878"
        ]
      }
    }
  }
}

The above nat configuration will result in the USG having the following iptables rules configured:

root@MainRouter:~# iptables -t nat -L PREROUTING -n --line-numbers | column -t
Chain  PREROUTING            (policy  ACCEPT)
num    target                prot     opt      source     destination
1      MINIUPNPD             all      --       0.0.0.0/0  0.0.0.0/0
2      UBNT_PFOR_DNAT_HOOK   all      --       0.0.0.0/0  0.0.0.0/0
3      VYATTA_PRE_DNAT_HOOK  all      --       0.0.0.0/0  0.0.0.0/0
4      DNAT                  tcp      --       0.0.0.0/0  !172.16.1.1  tcp  dpt:53  /*  NAT-1  tcp_udp  */  to:172.16.1.1:6878
5      DNAT                  udp      --       0.0.0.0/0  !172.16.1.1  udp  dpt:53  /*  NAT-1  tcp_udp  */  to:172.16.1.1:6878

2. Log back into the Unifi WebUI and execute another 'force provision' like we did previously.

3. Using the same tcpdump command, you can verify that the DNS query is no longer send unencrypted to Google's DNS servers but, instead intercepted and DNAT'ed to your dnscrypt-proxy daemon.

That's all for now! Future updates to this post may include further configuration options for dnscrypt-proxy and improve the IPv6 support for DNS resolution (currently that's not entirely working for my local network).  I'll also be looking into providing a failover configuration in the event that dnscrypt-proxy fails.

Update April 18, 2020:

Just a quick notice, when your USG upgrades firmware versions, it will blow away the SysVinit script at /etc/init.d/dnscrypt-proxy and restart the USG. The result is that if you have the entire above configuration, all DNS resolution will break. You can manually fix this by SSH'ing to the USG and running /config/dnscrypt-proxy/dnscrypt-proxy -service install && service dnscrypr-proxy start.

I'm looking into a permanent/automated recovery configuration and will update this post as I figure out a path forward.

Enjoy!

unsplash-logoChris Ried