270922 (3) [Avatar] Offline
#1
the etcdctl command is used in chapter 11 but there is no explanation as to how to set it up so it behaves like kubectl, i.e run it locally and see etcd registry entries on google cloud
Marko Lukša (65) [Avatar] Offline
#2
I included the listings showing the contents of etcd simply to show how the metadata is actually stored. Because resource manifests are stored as JSON documents, it's not too interesting to dig around etcd yourself, so I didn't want to put too much emphasis on how to do that.

Also, Google Container Engine doesn't allow direct access to etcd (and in general, tools should always access the cluster metadata through the API server, never directly).
nikolauz (27) [Avatar] Offline
#3
There should be a note about etcdctl command. I haven't found it.
Now (MEAP 10) it's not clear if that command is installed or not with kubernetes.
Thank you.
Marko Lukša (65) [Avatar] Offline
#4
I agree. I'll see if I can squeeze it in, but that's not likely, as the pages have already been typeset.

Anyway, running etcdctl is a bit involved. It's not bundled with Kubernetes, so you'll need to install it yourself. And you need to do that on the master node running etcd. On GKE, that's not possible (they don't allow direct access to etcd in any way). It is possible on minikube, though. The way I do it is:

1. minikube ssh
2. curl -L https://github.com/coreos/etcd/releases/download/v3.2.10/etcd-v3.2.10-linux-amd64.tar.gz | gunzip -c | tar xO etcd-v3.2.10-linux-amd64/etcdctl > etcdctl && chmod +x etcdctl
3. export ETCDCTL_API=3
4. ./etcdctl get /registry --prefix=true --keys-only

Kubernetes no longer stores plain text JSON in etcd, so you'll not be able to look at the contents of individual resources, but you can at least still see the hierarchy of the stored keys.

563812 (1) [Avatar] Offline
#5
Got this error by following the steps: Error: context deadline exceeded
SHOAIB R KHAN (2) [Avatar] Offline
#6
In the newer versions, ETCD servers are running with TLS enabled by default.


From the master node, you can confirm this by verifying how apiserver is communicating with etcd

ps -ef | grep apiserver

apiserver --advertise-address=192.168.122.243 --etcd-servers=https://192.168.122.243:2379,https://192.168.122.142:2379,https://192.168.122.176:2379 --etcd-cafile=/etc/ssl/etcd/ssl/ca.pem --etcd-certfile=/etc/ssl/etcd/ssl/node-server1.pem --etcd-keyfile=/etc/ssl/etcd/ssl/node-server1-key.pem


Now that you are sure ETCD has advertised its members on secure HTTPS endpoints and also have highlighted above the Private Key and Certificate file it's referencing you could use etcdctl easily by just appending key and cert files.
[apizone@server1 tmp]$ etcdctl --key /tmp/ca-key.pem --cert /tmp/ca.pem member list

8959ed642c954627, started, etcd2, https://192.168.122.142:2380, https://192.168.122.142:2379
ac9bc72680fa357d, started, etcd3, https://192.168.122.176:2380, https://192.168.122.176:2379
cb57f3947153c7c7, started, etcd1, https://192.168.122.243:2380, https://192.168.122.243:2379


PS: I have copied the key (ca-key.pem) and cert (ca.pem) files from /etc/ssl/etcd/ssl to /tmp and assigned rights to local user as it's usually not owned by root or kube user depending on how you setup your cluster.
Also you could set it up as part of environment under /etc/environment to avoid referencing the keys & cert every time