Note:
This is definitely not the easiest way to serve static content on the web but we’ll be pedantic, mostly cause it’s fun.
So I recently decided to move management of this site onto Kubernetes and I’d been meaning to try out DO’s Kubernetes offering and I have to say - it works great!
In this post, we’ll look at a bunch of different things that will all come together to give us the development experience we might’ve come to expect in this new world.
Things we’ll cover:
- Launching a managed Kubernetes cluster on Digitalocean(Coming soon)
- Running a lot of dependencies in the form of Kubernetes controllers to perform the following tasks
- Managing DNS records through the DO Domains API via kubernetes-incubator/external-dns
- Automatically fetching SSL certificates through Let’s Encrypt(ACME) via jetstack/cert-manager
- Serving static content via kubernetes/ingress-nginx
There are some smaller edge cases that I won’t spend time on, but are covered by the code:
- Redirecting a subdomain to the apex
- Redirecting http -> https
- …
TL;DR
git clone https://github.com/kasisnu/hugo-dok8s-letsencrypt
cd hugo-dok8s-letsencrypt
kubectl create secret generic digitalocean-secrets
kubectl edit secret digitalocean-secret # remember: base64 encoded values
kubectl apply -f . --recursive
# Pray everything worked
# If it didn't, read the long version and try again
The long version
Soft assumptions I’ve made:
- You’re using a K8s cluster where you have elevated permissions
- You can create new namespaces
- You can install controllers in these namespaces
- You’re managing DNS records for your domain through DO
- You’re aware of what we’re doing but not about how we’re doing it
Please note that these are not hard requirements. If you don’t have elevated permissions or you’re using a different DNS provider, you’ll have to make some changes to the manifests but things should “just work.” If you’re having issues, reach out to me and I’ll do my best to help.
First things first, make sure you have the repository available locally.
git clone https://github.com/kasisnu/hugo-dok8s-letsencrypt
cd hugo-dok8s-letsencrypt
External DNS
External DNS is a project meant to help you map K8s services and ingresses directly to an external DNS service provider like DNSimple, DigitalOcean, and so many others, which can easily be configured using annotations.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: www # maps to www.your-fancy-subdomain.tld
namespace: default
...
Before we start, let’s put our DigitalOcean API token someplace safe*.
kubectl create secret generic digitalocean-secrets
kubectl edit secret digitalocean-secret # remember: base64 encoded values
# double check the value
# the key "DO_TOKEN" is important
kubectl get secret digitalocean-secrets -o yaml
apiVersion: v1
data:
DO_TOKEN: <base-64-encoded-secret>
kind: Secret
...
If this is your first encounter with secrets, check out https://kubernetes.io/docs/concepts/configuration/secret/.
Next, follow the instructions from the docs for your service provider.
Alternativly, if you’re sure about using DigitalOcean Domains, run the following:
pushd external-dns
The more important parts live around:
# external-dns.yaml | EDIT ME
- name: external-dns
image: registry.opensource.zalan.do/teapot/external-dns:latest
args:
- --source=service # work with services
- --source=ingress # work with ingresses
- --domain-filter=<change-me.com> # [optional] limit ourselves to a specfic domain
- --provider=digitalocean # for now, only work with DO
- --registry=txt
- --log-level=debug # [optional] print a little more debugging information
Now apply the manifests.
kubectl apply -f .
popd
Inspect what just happened:
kubectl get pods -l'app=external-dns'
NAME READY STATUS RESTARTS AGE
external-dns-7c4bf9ffb4-tqkwb 1/1 Running 0 11d
If you think something isn’t working, debug it with:
# Inspect secret
kubectl get secret digitalocean-secrets -o yaml
# Inspect logs
kubectl logs -lapp=external-dns
# Bounce/restart external dns pods
kubectl delete pods -lapp=external-dns
# Fix the issue and start over
Cert Manager
Cert manager does exactly what it sounds like. It will talk ACME and get us a set of TLS certificates, store them as secrets, and will also renew them periodically.
```yaml`
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: www
namespace: default
annotations:
certmanager.k8s.io/cluster-issuer: letsencrypt-prod
# It is possible to run multiple cluster issuers which can be individually configured
# for more flexibility
spec:
tls:
- hosts:
- domain.tld
- www.domain.tld secretName: my-fancy-tls-certificate # this is where the certificates are stored …
To get started, follow the instructions from the [docs](https://cert-manager.readthedocs.io/en/latest/getting-started/index.html).
Alternativly, if you're in a hurry:
```sh
pushd cert-manager
The more important parts live around:
# cluster-issuer.yaml | EDIT ME
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: <change-me@someexample.org>
privateKeySecretRef:
name: letsencrypt-prod
dns01:
providers:
- name: dns
digitalocean:
tokenSecretRef:
name: digitalocean-secrets
key: DO_TOKEN # this has to be the same key as the secret above
Now apply the manifests.
kubectl apply -f .
popd
Ingress Nginx
To get started, follow the instructions from the docs.
Alternativly, if you’re in a hurry:
pushd ingress-nginx
kubectl apply -f .
popd
Now the cluster should look a lot different than before we started. Take a moment to inspect the cluster. List namespaces, pods, services, deployments, secrets, certificates. Building a good mental model is going to be important so we can debug effectively in the future.
Run Hugo
Okay, so I said we were going to serve static content. It’s time to finally do that. That is covered in Part 2 but maybe we should take a break.