You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
|
2 years ago | |
---|---|---|
control-api | 2 years ago | |
host-cluster | 2 years ago | |
in-cluster | 2 years ago | |
.gitignore | 2 years ago | |
README.adoc | 2 years ago | |
kubeconfig.yaml | 2 years ago | |
values.yaml | 2 years ago |
README.adoc
vcluster on OpenShift
See also vcluster docs.
Notes
-
Issues with vcluster on OpenShift
-
Images do not run properly as non-root - workaround with multiple
emptyDir
mounts. -
Default K3s CoreDNS deployment doesn’t work on OpenShift Host Cluster → Custom deployment in
in-cluster/coredns.yaml
.
-
-
Re-encrypt route only works with OIDC auth, not with certificate auth
Installation
-
Create OpenShift Project
-
Create and configure vcluster
vcluster create vcluster-1 -n tobru-vcluster-poc -f values.yaml vcluster connect vcluster-1 --namespace tobru-vcluster-poc kubectl --kubeconfig=$(pwd)/kubeconfig.yaml apply -f in-cluster/
-
Install Route
CA=$(kubectl --kubeconfig=$(pwd)/kubeconfig.yaml config view -o jsonpath='{.clusters[].cluster.certificate-authority-data}' --flatten | base64 -d) yq -i e ".spec.tls.destinationCACertificate = \"$CA\"" host-cluster/route.yaml oc apply -f host-cluster/route.yaml
-
Configure kubeconfig for vcluster access
kubectl --kubeconfig=$(pwd)/kubeconfig.yaml config unset clusters.local.certificate-authority-data kubectl --kubeconfig=$(pwd)/kubeconfig.yaml config set clusters.local.server https://vcluster-poc.apps.cloudscale-lpg-1.appuio.cloud kubectl --kubeconfig=$(pwd)/kubeconfig.yaml config set-credentials oidc \ --exec-api-version=client.authentication.k8s.io/v1beta1 \ --exec-command=kubectl \ --exec-arg=oidc-login \ --exec-arg=get-token \ --exec-arg=--oidc-issuer-url=https://id.dev.appuio.cloud/auth/realms/appuio-cloud-dev \ --exec-arg=--oidc-client-id=tobru-vcluster-test \ --exec-arg=--oidc-client-secret=63410f24-0721-447b-a290-4b0169c414e0 kubectl --kubeconfig=$(pwd)/kubeconfig.yaml config set-context Default --user=oidc
🚀
Background information: OIDC Authentication
vcluster config
vcluster:
baseArgs:
[...]
- --kube-controller-manager-arg=controllers=*,-nodeipam,-nodelifecycle,-persistentvolume-binder,-attachdetach,-persistentvolume-expander,-cloud-node-lifecycle
- --kube-apiserver-arg=oidc-client-id=tobru-vcluster-test
- --kube-apiserver-arg=oidc-groups-claim=groups
- --kube-apiserver-arg=oidc-issuer-url=https://id.dev.appuio.cloud/auth/realms/appuio-cloud-dev
- --kube-apiserver-arg=oidc-username-claim=email
kubectl plugin
# kubectl oidc-login setup --oidc-issuer-url=https://id.dev.appuio.cloud/auth/realms/appuio-cloud-dev --oidc-client-id=tobru-vcluster-test --oidc-client-secret=63410f24-0721-447b-a290-4b0169c414e0
authentication in progress...
## 2. Verify authentication
You got a token with the following claims:
{
"exp": 000,
"iat": 000,
"auth_time": 000,
"jti": "XXX",
"iss": "https://id.dev.appuio.cloud/auth/realms/appuio-cloud-dev",
"aud": "tobru-vcluster-test",
"sub": "UUID",
"typ": "ID",
"azp": "tobru-vcluster-test",
"nonce": "XXX",
"session_state": "XXX",
"at_hash": "XXX",
"acr": "1",
"sid": "XXX",
"email_verified": true,
"name": "Tobias Brunner",
"groups": [
"admin"
],
"preferred_username": "tobias.brunner",
"given_name": "Tobias",
"family_name": "Brunner",
"email": "tobias.brunner@vshn.net"
}
## 3. Bind a cluster role
Run the following command:
kubectl create clusterrolebinding oidc-cluster-admin --clusterrole=cluster-admin --user='XXX'
## 4. Set up the Kubernetes API server
Add the following options to the kube-apiserver:
--oidc-issuer-url=https://id.dev.appuio.cloud/auth/realms/appuio-cloud-dev
--oidc-client-id=tobru-vcluster-test
## 5. Set up the kubeconfig
Run the following command:
kubectl config set-credentials oidc \
--exec-api-version=client.authentication.k8s.io/v1beta1 \
--exec-command=kubectl \
--exec-arg=oidc-login \
--exec-arg=get-token \
--exec-arg=--oidc-issuer-url=https://id.dev.appuio.cloud/auth/realms/appuio-cloud-dev \
--exec-arg=--oidc-client-id=tobru-vcluster-test \
--exec-arg=--oidc-client-secret=63410f24-0721-447b-a290-4b0169c414e0
## 6. Verify cluster access
Make sure you can access the Kubernetes cluster.
kubectl --user=oidc get nodes
You can switch the default context to oidc.
kubectl config set-context --current --user=oidc
You can share the kubeconfig to your team members for on-boarding.
# kubectl --kubeconfig ./kubeconfig.yaml config set-credentials oidc \
--exec-api-version=client.authentication.k8s.io/v1beta1 \
--exec-command=kubectl \
--exec-arg=oidc-login \
--exec-arg=get-token \
--exec-arg=--oidc-issuer-url=https://id.dev.appuio.cloud/auth/realms/appuio-cloud-dev \
--exec-arg=--oidc-client-id=tobru-vcluster-test \
--exec-arg=--oidc-client-secret=63410f24-0721-447b-a290-4b0169c414e0
# kubectl --kubeconfig ./kubeconfig.yaml --user=oidc get po -A
Keycloak client
-
"Access Type" confidential
-
Valid Redirect URIs: http://localhost:18000 http://localhost:8000 (they are for the kubelogin plugin)