diff --git a/README.md b/README.md index cc4953c..6c4fcde 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,6 @@ git submodule init git submodule update ``` - ## Installation ```sh @@ -48,7 +47,53 @@ vi default.nix # add nodes and ip:s, etc. ../../bin/deploy.sh cluster-1 ``` -## TODO +## etcd clustering guide + +Using fish: + +1. Install standard nixos k8s +2. Run the `init-admin-kubeconfig.sh` script in the `k8s-charts` repo. +3. Snapshot the etcd database: `etcdctl snapshot save (date --iso-8601).etcd` +4. Add the external interface to the etcd peers: + ``` + etcdctl member list + etcdctl member update [id] --peer-urls=https://[extenal ip]:2380 + etcdctl member list + ``` +4. In the nixos cluster config, enable clustering for all nodes: + ``` + etcdNodes = {}; + + features.k8s.etcdCluster = { + enable = true; + existing = false; # true for master node! + nodes = etcdNodes; + }; + ``` +5. Add the next node `kN-1` to the cluster (on the master node): + ``` + etcdctl member add kN-1 --peer-urls=https://[ip2]:2380 + ``` +6. ssh into `kN-1` and run the `etcd-join-cluster` script. +7. Add the next node `kN-2` to the cluster (on the master node): + ``` + etcdctl member add kN-2 --peer-urls=https://[ip2]:2380 + ``` +8. ssh into `kN-2` and run the `etcd-join-cluster` script. +9. Zap (ctrl-c) the running etcd:s on kN-1 and 2. +10. Uncomment etcdNodes and set existing to true for all, and redeploy cluster + ``` + etcdNodes = { + kN-0 = "https://[ip1]:2380"; + kN-1 = "https://[ip2]:2380"; + kN-2 = "https://[ip3]:2380"; + }; + + features.k8s.etcdCluster = { + enable = true; + existing = true; + nodes = etcdNodes; + }; + ``` +11. Check logs, restart etcd on nodes if necessary, etc. -* Self-provisioning NixOs iso image -* Terraform setup for provisioning a cluster