138 lines
3.1 KiB
Markdown
138 lines
3.1 KiB
Markdown
# GitOps for tbrnt k3s hosting
|
|
|
|
[](https://drone.tbrnt.ch/tobru/gitops-tbrnt)
|
|
|
|
## Repo structure
|
|
|
|
* Each subdirectory is a namespace
|
|
* `_apps` is the meta directory for Argo CD apps
|
|
* Another private repo contains stuff in a more
|
|
approachable format, f.e. for dealing with
|
|
updating sealed-secrets: `gitops-tbrnt-private`
|
|
* `_tests` contains some Open Policy Agent rego
|
|
files which are used in the Drone CI pipeline
|
|
to validate configuration.
|
|
|
|
## Usage
|
|
|
|
### Argo CD
|
|
|
|
#### Access
|
|
|
|
Either
|
|
|
|
`sudo -E kubefwd svc -n argocd` and then https://argocd-server/
|
|
|
|
or
|
|
|
|
`kubectl port-forward svc/argocd-server -n argocd 8080:443` and
|
|
then https://localhost:8080/
|
|
|
|
#### CLI
|
|
|
|
* `argocd login argocd-server`
|
|
* `argocd app list`
|
|
* `argocd app sync <name>`
|
|
|
|
### Kubeseal (Sealed Secrets)
|
|
|
|
See README of apps. Basically:
|
|
|
|
```
|
|
kubeseal --controller-namespace sealed-secrets -o yaml -n MYNS < ../../gitops-tbrnt-private/MYNS/MYSECRET.yaml > MYSECRET-secret.yaml
|
|
```
|
|
|
|
## Bootstrap GitOps
|
|
|
|
After installing k3s, do:
|
|
|
|
```
|
|
# install Argo CD
|
|
kubectl create ns argocd
|
|
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
|
|
kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2
|
|
argocd login argocd-server
|
|
|
|
# Restore Sealed Secrets secret key
|
|
kubectl create ns sealed-secrets
|
|
kubectl apply -f ../gitops-tbrnt-private/sealed-secrets/master-key.yaml
|
|
|
|
# Instantiate Argo Root App
|
|
kubectl apply -f _apps/apps.yaml
|
|
|
|
# Let Argo CD do it's job
|
|
argocd app sync apps
|
|
argocd app sync sealed-secrets
|
|
argocd app sync -l app.kubernetes.io/instance=apps
|
|
```
|
|
|
|
TODO:
|
|
* Restore PVCs via K8up
|
|
|
|
## k3s on Alpine
|
|
|
|
### Installing: Alpine
|
|
|
|
Basically follow the [Alpine wiki](https://wiki.alpinelinux.org/wiki/Installation).
|
|
|
|
Then install prerequisites and some essential packages:
|
|
|
|
```
|
|
apk add \
|
|
vim \
|
|
iptables \
|
|
wireguard-virt \
|
|
bash \
|
|
curl
|
|
```
|
|
|
|
Needs `community` repo enabled in `/etc/apk/repositories`.
|
|
|
|
Tweak Sysctl in `/etc/sysctl.conf`:
|
|
|
|
```
|
|
fs.inotify.max_user_instances = 8192
|
|
fs.inotify.max_user_watches = 524288
|
|
```
|
|
|
|
### Installing: k3s
|
|
|
|
Via [k3sup](https://github.com/alexellis/k3sup):
|
|
|
|
```
|
|
k3sup install \
|
|
--ip=185.95.218.11 \
|
|
--user=root \
|
|
--local-path=~/.kube/config_knurrli2 \
|
|
--sudo=false \
|
|
--k3s-extra-args='--tls-san knurrli.tobrunet.ch --cluster-cidr 10.44.0.0/16 --flannel-backend wireguard'
|
|
```
|
|
|
|
### Helpful infos
|
|
|
|
*Paths*
|
|
* Volumes: `/var/lib/rancher/k3s/storage/`
|
|
* Config: `/etc/rancher/k3s/`
|
|
* Manifests: `/var/lib/rancher/k3s/server/manifests/`
|
|
|
|
*Links*
|
|
* https://rancher.com/docs/k3s/latest/en/advanced/#additional-preparation-for-alpine-linux-setup
|
|
* https://github.com/rancher/k3s/issues/660
|
|
|
|
## Configure Wireguard
|
|
|
|
`/etc/network/interfaces`
|
|
|
|
```
|
|
auto wg0
|
|
iface wg0 inet static
|
|
address 10.42.42.16
|
|
netmask 255.255.255.0
|
|
pre-up ip link add dev wg0 type wireguard
|
|
pre-up wg setconf wg0 /etc/wireguard/wg0.conf
|
|
post-up ip route add 10.42.42.0/24 dev wg0
|
|
post-down ip link delete dev wg0
|
|
```
|
|
|
|
* https://wiki.alpinelinux.org/wiki/Configure_a_Wireguard_interface_(wg)
|