Public GitOps repository for showing how I run my services on k3s
https://tobru.ch/
Tobias Brunner
6d2e8c9c0e
This is really bad and it has stolen 3h from my live. I should have known it! |
||
|---|---|---|
| _apps | ||
| _test | ||
| argocd | ||
| cert-manager | ||
| drone | ||
| graphs | ||
| influxdb | ||
| ioteer | ||
| ipapi | ||
| jitsi | ||
| k8up | ||
| loki | ||
| monitoring | ||
| mosquitto | ||
| owntracks | ||
| pylokid | ||
| renovate | ||
| sealed-secrets | ||
| stakater-reloader | ||
| statping | ||
| tobru-ch | ||
| .drone.yml | ||
| README.md | ||
| renovate.json | ||
GitOps for tbrnt k3s hosting
Repo structure
- Each subdirectory is a namespace
_appsis the meta directory for Argo CD apps- Another private repo contains stuff in a more
approachable format, f.e. for dealing with
updating sealed-secrets:
gitops-tbrnt-private _testscontains some Open Policy Agent rego files which are used in the Drone CI pipeline to validate configuration.
Usage
Argo CD
Access
Either
sudo -E kubefwd svc -n argocd and then https://argocd-server/
or
kubectl port-forward svc/argocd-server -n argocd 8080:443 and
then https://localhost:8080/
CLI
argocd login argocd-serverargocd app listargocd app sync <name>
Kubeseal (Sealed Secrets)
See README of apps. Basically:
kubeseal --controller-namespace sealed-secrets -o yaml -n MYNS < ../../gitops-tbrnt-private/MYNS/MYSECRET.yaml > MYSECRET-secret.yaml
Bootstrap GitOps
After installing k3s, do:
# install Argo CD
kubectl create ns argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2
argocd login argocd-server
# Restore Sealed Secrets secret key
kubectl create ns sealed-secrets
kubectl apply -f ../gitops-tbrnt-private/sealed-secrets/master-key.yaml
# Instantiate Argo Root App
kubectl apply -f _apps/apps.yaml
# Let Argo CD do it's job
argocd app sync apps
argocd app sync sealed-secrets
argocd app sync -l app.kubernetes.io/instance=apps
TODO:
- Restore PVCs via K8up
k3s on Alpine
Installing: Alpine
Basically follow the Alpine wiki.
Then install prerequisites and some essential packages:
apk add \
vim \
iptables \
wireguard-virt \
bash \
curl
Needs community repo enabled in /etc/apk/repositories.
Tweak Sysctl in /etc/sysctl.conf:
fs.inotify.max_user_instances = 8192
fs.inotify.max_user_watches = 524288
Installing: k3s
Via k3sup:
k3sup install \
--ip=185.95.218.11 \
--user=root \
--local-path=~/.kube/config_knurrli2 \
--sudo=false \
--k3s-extra-args='--tls-san knurrli.tobrunet.ch --cluster-cidr 10.44.0.0/16 --flannel-backend wireguard'
Helpful infos
Paths
- Volumes:
/var/lib/rancher/k3s/storage/ - Config:
/etc/rancher/k3s/ - Manifests:
/var/lib/rancher/k3s/server/manifests/
Links
- https://rancher.com/docs/k3s/latest/en/advanced/#additional-preparation-for-alpine-linux-setup
- https://github.com/rancher/k3s/issues/660
Configure Wireguard
/etc/network/interfaces
auto wg0
iface wg0 inet static
address 10.42.42.16
netmask 255.255.255.0
pre-up ip link add dev wg0 type wireguard
pre-up wg setconf wg0 /etc/wireguard/wg0.conf
post-up ip route add 10.42.42.0/24 dev wg0
post-down ip link delete dev wg0