Jake Teton‑Landis

Perfection enthusiast and byte craftsman splitting time between Miami, FL and New York, NY.

Interested in human productivity, big levers, and design.

Github • Mastodon
Twitter • LinkedIn

Cozy Server

October - November 2019

I just finished putting together a new home server. It's tiny. It's quiet. It's simple. It's got a super-fast SSD, six cores, and a bunch of RAM.

My goal is to never be frustrated by this system -- that's what makes it cozy.

Table of Contents

Why build now?

Winter 2019 is a great season to build a new cozy home server. AMD, the x86 CPU underdog, released a new generation of processors in July: the Zen 2 architecture. AMD has been the "value" choice for home system builders for a while, but with Zen 2, AMD has a considerable multi-thread performance lead while having lower prices than Intel.

Hardware

This is a 7-liter mini-ITX build that will comfortably fit in your TV media center. Overall, my part choices are spendy. There's a few places one could cut down that price tag.

Type Item Price
CPU AMD Ryzen 5 3600 3.6 GHz 6-Core Processor \$194
Motherboard Gigabyte X570 I AORUS PRO WIFI Mini ITX AM4 Motherboard \$219
Memory Corsair Vengeance LPX 32 GB (2 x 16 GB) DDR4-3200 Memory \$140
Storage Samsung 970 EVO Plus 2 TB M.2-2280 NVME Solid State Drive \$443
Video Card VisionTek Radeon HD 6350 1 GB Video Card \$68
Case Silverstone ML05B HTPC Case \$64
Power Supply Corsair SF 450 W 80+ Platinum Certified Fully Modular SFX Power Supply \$100
Total \$1228
My [PCPartPicker Part List](https://pcpartpicker.com/list/7bnjp8) has every component except for the video card - the one I bought from Amazon isn't listed on PCPartPicker.

CPU: AMD Ryzen 5 3600

This was the lowest-power 6-core Zen 2 part available. I wanted six cores because my application load sometimes includes multi-threaded game servers and media transcoders.

None of the Zen 2 parts have onboard video support, so choosing this CPU means we have to supply a PCI video card.

Motherboard: X570 AORUS PRO WIFI Mini ITX

Mini ITX is the smallest size of consumer motherboard, and it's our choice for this small server build.

At around \$200, this motherboard is on the expensive side. There are B450 ITX motherboards half the price with nominal compatibility with our CPU choice, but older boards might need a BIOS upgrade with similar features for our purposes, but I heard rumors that Linux on Ryzen 3000 doesn't play well with the older motherboards. Plus, some older motherboards need a BIOS upgrade before they can be used with the newer AMD CPUs. Annoying.

Storage: Samsung 970 EVO Plus 2 TB M.2 NVME

NVME is an amazing technology - it's crazy that a chip the size of a pack of gum can hold 2TB of the fastest storage around.

At $450, this is the single most expensive part on our list, and it's also the hardest to justify. The Sabrent Rocket 2TB NVME has similar performance at half the price ($250), and a 2TB spinning disk is a fifth the price at \$83. I picked Samsung for peace-of-mind. Back in the day, cheap SSDs would fail randomly so it was worth buying a premium part. Supposedly things work better across the board these days.

Video Card: VisionTek Radeon HD 6350 1 GB Video Card

This was the smallest-looking AMD video card I could find on Amazon when scrounging around. It needs to be a "low profile" half-height card to fit in our small case. AMD works much better with Linux than Nvidia - Nvidia is an enemy of your freedom. Plus, AMD drivers are in the kernel so there's less mucking about required.

Unfortunately, it turns out this video card is so old, it can't be put into a low-power state when it's not in use! That means it both draws power all the time, which is bad for the environment, and it produces heat, which leads to a noisy spinning CPU fan! I do not recommend this card! It's bad. I haven't found a good solution other than removing the card.

Case: Silverstone ML05B HTPC Case

My original intended case was the Inwin Chopin, an even-smaller case with no space for a graphics card. However, my motherboard didn't fit into the Chopin because the Chopin uses some strange bulbous stand-offs.

So, I rage-bought this HTPC case from Amazon! It looks nice in person, but if I did it again, I'd get a case with space for a two-slot PCI card, so I could upgrade to a larger graphics card... one with a more modern GPU that could enter a low power state.

Power Supply: Corsair SF 450W 80+ Platinum SFX

Like "ITX" is a small type of motherboard, "SFX" is small size of power supply. Corsair is a well-respected manufacturer. This is their lowest-wattage SFX (small-size) power supply that has fully-modular cables.

The Platinum efficiency rating means that this power supply will save me about \$32 a year - not too much. But since this computer will be on 24 hours a day for years, I wanted to get a well-rated power supply that would last.

Software

cozy.systems home page Heimdall dashboard

The cozy server runs Ubuntu 18.04.3. I manage configuration and set-up using Chef. The only aspects of the software stack that aren't managed by Chef are a fallback, local-login-only user account, and the initial system installation.

I didn't do anything fancy with partitioning like use LUKS or LVM or ZFS - I'm just using ex4 with a single partition. Plus an EFI boot partition.

Chef

I use Chef to manage system configuration. Max Burkhardt and I managed our shared home server with knife-solo for years. Although we've moved apart, we continue to use the same recipes for our personal servers. Our current repo is private, but you can find a much earlier iteration at justjake/chopped.

I find Chef delightful for hobby systems - I can write my system descriptions in Ruby, the most fun language, but I don't need to worry about the whole thing turning into a dynamically-typed Chef spaghetti disaster because the scope is so small - just one or two servers.

Our general philosophy:

  • Every recipe can run as up (install and configure) or down (stop, uninstall, and clean up).
  • Install and run services with Docker. This simplifies many management tasks and keeps the base system clean. Many of our Docker images come from the Linuxserver.io community.
  • Avoid making new cookbooks. What's the point?
  • Avoid Chef features like roles and attributes unless strictly necessary for variation between systems. Perform attribute lookups through a helper class for typo-safety.

Without further ado, here's the Chef role for the cozy server:

name "cozy_systems_app_server"
description "App server for the cozy.systems network"

default_attributes(
    'base' => {
        'sysadmin_users_data_bag' => 'cozy_users',
    },
    'armada' => {
        'enabled_recipes' => {
            'certbot' => 'up',
            'unifi' => 'up',
            'netdata' => 'up',
            'qbittorrent' => 'up',
            'sonarr' => 'up',
            'radarr' => 'up',
            'emby' => 'up',
            'dashboard' => 'up',
            'dyn_dns' => 'up',
        },
        'directories' => {
            'downloads' => '/home/media/downloads'
        },
        'domain_suffix' => 'cozy.systems',
        'local_network' => '192.168.1.0/24',
        'route53_zone_id' => 'Z...' # removed for privacy
    }
)

run_list 'recipe[armada]'

See those 'enabled_recipes'? That's what I'm running on the server! Let's break it down.

Certbot (letsencrypt)

Certbot is a package for automating Let's Encrypt certificate renewal. We use a single wildcard certificate for the whole *.cozy.systems domain to encrypt all the things. Yay HTTPS!

Our recipe creates a wrapper script on the root system that wraps the certbot/dns-route53 Docker image , which lets us renew by fiddling AWS Route53 DNS.

Unifi

I run my home network on Ubiquiti networks Unifi wifi gear. The wireless access points are powerful and very reliable for home use, but managing them does require running a Unifi "controller" server.

We install the Unifi server from the linuxserver/unifi docker image. Unifi uses a bunch of ports, we do a bunch of host-port mapping in Docker, plus some Nginx magic to make everything work:

# in site_cookbooks/armada/recipes/unifi.rb
# ...
docker_container 'unifi' do
  action enabled ? [:run] : [:stop, :delete]
  restart_policy 'unless-stopped'
  repo 'linuxserver/unifi'
  tag docker_tag
  log_driver 'syslog'
  memory 1024 * 1024 * 500
  # Tons of ports for Unifi access point management.
  # Access points need to connect it to register & download
  # configuration. No idea why it needs so many, though...
  port %W(
    #{armada.claim_port(3478, 'unifi')}:3478/udp
    #{armada.claim_port(10001, 'unifi')}:10001/udp
    #{armada.claim_port(8080, 'unifi')}:8080
    #{armada.claim_port(8081, 'unifi')}:8081
    #{armada.claim_port(8443, 'unifi')}:8443
    #{armada.claim_port(8843, 'unifi')}:8843
    #{armada.claim_port(8880, 'unifi')}:8880
    #{armada.claim_port(6789, 'unifi')}:6789
  )
  env [
    "PUID=#{unifi_uid}",
    "GUID=#{unifi_uid}"
  ]
  volumes [
    '/home/unifi:/config'
  ]
end

# chopped_nginx is our nginx-in-chef DSL. We prefer this over needing
# to make a zillion template Nginx configuration files.
# chopped_nginx is open source at
# https://github.com/justjake/chopped/tree/master/site-cookbooks/chopped_nginx
chopped_nginx_http 'unifi' do
  action enabled ? :create : :delete
  config do
    server do
      server_name armada.domain('wifi')
      server_name armada.remote_domain('wifi')
      listen 443, :ssl
      location '/' do |l|
        l.proxy_ssl_verify :off
        l.proxy_http_version '1.1'
        l.proxy_set_header :Upgrade, '$http_upgrade'
        l.proxy_set_header :Connection, :upgrade
        armada.proxy_pass(l, "https://127.0.0.1:#{web_interface_port}/")
      end
    end

    server do
      server_name armada.domain('wifi')
      server_name armada.remote_domain('wifi')
      listen 80
      location '/' do |l|
        armada.proxy_pass(l, "http://127.0.0.1:#{notify_port}/")
      end
    end
  end
end

Netdata

Netdata shows you graphs about the system. I use it to figure out which Docker container is pegging the CPU, and to check on motherboard temperature sensors.

netdata screenshot

We install it via Docker from the official netdata/netdata repo. To allow netdata to query the host system about containers securely, we also install a docker socket proxy.

By default Netdata scans your system for about a million different things. It comes with a bunch of annoying health-check alarms too. Most of our customization of Netdata is disabling all this stuff.

Beware! By default netdata sends home usage statistics, and encourages you to register your server in their public remote instance browser! This is creepy. If you consider installing Netdata, be sure to:

Here's the Chef templates I use to configure Netdata. Both files are placed in /etc/netdata -- you'll need to replace the ERB template bits as needed.

# netdata.conf
[global]
  hostname = <%= node['hostname'] %>

# Netdata comes with tons of alarms by default.
# Some are pretty crap for our use-case.
# Disable all.
[health]
  enabled = no

# Need to run custom registry to avoid data leak
# https://docs.netdata.cloud/docs/configuration-guide/#make-security-related-customizations
# https://docs.netdata.cloud/registry/#run-your-own-registry
[registry]
  enabled = yes
  registry to announce = https://<%= armada.domain('netdata') %>
  registry hostname = <%= node['hostname'] %>

# Netdata comes with a ton of plugins to scape info
# Plugins ending in *.d host sub-plugins and typically have their own config file.
#
# See /usr/libexec/netdata/plugins.d inside the container
# See /usr/lib/netdata/conf.d inside the container
# https://docs.netdata.cloud/docs/add-more-charts-to-netdata/#enabling-and-disabling-plugins
[plugins]
  # https://github.com/netdata/netdata/tree/master/collectors/python.d.plugin
  # We configure this to opt-in to plugins.
  python.d = yes

  # https://github.com/netdata/go.d.plugin
  # We don't care about any of the plugins hosted by go.d
  go.d = no

  # https://github.com/netdata/netdata/tree/master/collectors/node.d.plugin
  # We don't care about any of the plugins hosted by node.d
  node.d = no

  # https://github.com/netdata/netdata/tree/master/collectors/charts.d.plugin
  # Bash v4 runtime. Mostly superceded by Python
  charts.d = no

# python.d.conf
# netdata python.d.plugin configuration

# Enable / disable the whole python.d.plugin (all its modules)
enabled: yes

# ----------------------------------------------------------------------
# Enable / Disable python.d.plugin modules
#
# If "default_run" = "yes" the default for all modules is enabled (yes).
# Setting any of these to "no" will disable it.
#
# If "default_run" = "no" the default for all modules is disabled (no).
# Setting any of these to "yes" will enable it.
default_run: no

# Enable / Disable explicit garbage collection (full collection run). Default is enabled.
gc_run: yes

# Garbage collection interval in seconds. Default is 300.
gc_interval: 300

# Enable modules we actually want.
# Sensors: show temprature graphs
sensors: yes
w1sensor: yes

qBittorrent

qBittorrent is low-resource and has an HTTP API with good integrations for our other software. It's not flashy, but it works great. We install this from the linuxserver/qbittorrent docker repo.

qBittorrent screenshot

qBittorrent detects a naïve Nginx reverse-proxy as a CSRF attack for some reason. Here's how we work around that:

chopped_nginx_http 'qbittorrent' do
  action enabled ? :create : :delete
  config do
    server do
      server_name armada.domain('torrents')
      server_name armada.remote_domain('torrents')
      listen 443, :ssl
      location '/' do |l|
        # proxy mechanics
        l.proxy_pass        "http://#{backend_port}/"
        l.proxy_set_header  :Upgrade,           '$http_upgrade'
        l.proxy_set_header  :Connection,        :upgrade

        # shenanigans to pass qbt's CSRF protection (?)
        l.proxy_set_header  'X-Forwarded-Host',       backend_port
        l.proxy_set_header  'X-Forwarded-For',        '127.0.0.1'
        l.proxy_hide_header :Referer
        l.proxy_hide_header :Origin
        l.proxy_set_header  :Referer,                 "''";
        l.proxy_set_header  :Origin,                  "''";
        l.proxy_set_header  :Host,                    backend_port
        l.add_header        'X-Frame-Options',        "SAMEORIGIN"

        armada.protect(l)
      end
    end
  end
end

Sonarr & Radarr

Sonarr automates torrenting TV. Radarr automates torrenting movies. Put in the name of the thing you want, tell it the quality, and it'll go out to the torrent sites, find the right torrent for you, download it, and then move the finished files into a nicely organized directory structure. This is especially nice for episodic TV - once you subscribe to a show, new episodes show up on the filesystem soon after the show airs, ready to be watched!

Radarr screenshot

Sonarr and Radarr connect to qBittorrent over its HTTP API to manage downloads, and they also talk to the Emby API to index completed downloads in the media library.

Docker containers come from linuxserver/sonarr and linuxserver/radarr.

Emby

Emby is a media server and home streaming app. I mostly use it via the excellent Apple TV app, but playback works great in a browser, too.

Emby screenshot

We install Emby from the offical emby/embyserver repo. Be sure to follow the instructions to allow Emby to offload encoding work to your graphics hardware.

Dashboard (Heimdall)

Heimdall provides us with a nice homepage that links out to our other services.