TechContainerKubernetes

【容器化】Kubernetes二进制高可用部署3-Etcd高可用部署

, , , , ,
Views: 1000000
Words: 1.3k Read time: 6 mins.

零、目录

一、说明

本文将在三个虚拟机上都部署etcd服务，已达到一个三节点的etcd高可用集群。

二、etcd证书生成

ssl配置文件

注意点：alt_names的值指定为节点ip

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# 192.168.56.105节点执行
mkdir -p /etc/etcd/pki
cat << EOF > /etc/etcd/pki/etcd_ssl.cnf
[ req ]
req_extensions = v3_req
distinguished_name = req_distinguished_name

[ req_distinguished_name ]

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[ alt_names ]
IP.1 = 192.168.56.105
IP.2 = 192.168.56.106
IP.3 = 192.168.56.107
EOF

服务端证书

1
2
3
4
# 192.168.56.105节点执行
openssl genrsa -out /etc/etcd/pki/etcd_server.key 2048
openssl req -new -key /etc/etcd/pki/etcd_server.key -config /etc/etcd/pki/etcd_ssl.cnf -subj "/CN=etcd-server" -out /etc/etcd/pki/etcd_server.csr
openssl x509 -req -in /etc/etcd/pki/etcd_server.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -days 36500 -extensions v3_req -extfile /etc/etcd/pki/etcd_ssl.cnf -out /etc/etcd/pki/etcd_server.crt

客户端证书

1
2
3
4
# 192.168.56.105节点执行
openssl genrsa -out /etc/etcd/pki/etcd_client.key 2048
openssl req -new -key /etc/etcd/pki/etcd_client.key -config /etc/etcd/pki/etcd_ssl.cnf -subj "/CN=etcd-client" -out /etc/etcd/pki/etcd_client.csr
openssl x509 -req -in /etc/etcd/pki/etcd_client.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -days 36500 -extensions v3_req -extfile /etc/etcd/pki/etcd_ssl.cnf -out /etc/etcd/pki/etcd_client.crt

复制etcd证书到106、107虚拟机

1
2
3
# 192.168.56.106和192.168.56.107节点执行
mkdir -p /etc/etcd/pki
scp -r root@192.168.56.105:/etc/etcd/pki/ /etc/etcd

三、下载etcd并解压并把二进制文件复制到/usr/bin目录

1
2
3
4
# 所有节点执行
wget -O /usr/local/src/etcd-v3.4.13-linux-amd64.tar.gz https://github.com/etcd-io/etcd/releases/download/v3.4.13/etcd-v3.4.13-linux-amd64.tar.gz
tar -C /usr/local/src/ -zxvf /usr/local/src/etcd-v3.4.13-linux-amd64.tar.gz
cp /usr/local/src/etcd-v3.4.13-linux-amd64/etcd /usr/local/src/etcd-v3.4.13-linux-amd64/etcdctl /usr/bin/

四、创建etcd服务

创建etcd配置文件

etcd服务参数说明：

  • ETCD_NAME：etcd节点名称，每个节点都应不同，例如：etcd1、etcd2、etcd3
  • ETCD_DATA_DIR：etcd数据存储目录，例如：/etc/etcd/data
  • ETCD_LISTEN_CLIENT_URLS和ETCD_ADVERTISE_CLIENT_URLS：为客户端提供的服务监听URL地址，例如：https://192.168.56.105:2379
  • ETCD_LISTEN_PEER_URLS和ETCD_INITIAL_ADVERTISE_PEER_URLS：为本etcd集群其他节点提供的服务监听URL地址，例如：https://192.168.56.105:2380
  • ETCD_INITIAL_CLUSTER_TOKEN：etcd集群名称，例如：etcd-cluster
  • ETCD_INITIAL_CLUSTER：etcd集群各节点的endpoint列表
  • ETCD_INITIAL_CLUSTER_STATE：初始集群状态，新建集群时设置为“new”，集群已存在时设置为“existing”

ca证书参数说明：

  • ETCD_CERT_FILE：etcd服务端CA证书crt文件全路径
  • ETCD_KEY_FILE：etcd服务端CA证书key文件全路径
  • ETCD_TRUSTED_CA_FILE：CA根证书文件全路径
  • ETCD_CLIENT_CERT_AUTH：是否启用客户端证书认证
  • ETCD_PEER_CERT_FILE：etcd集群各节点相互认证使用的CA证书crt文件全路径
  • ETCD_PEER_KEY_FILE：etcd集群各节点相互认证使用的CA证书key文件全路径
  • ETCD_PEER_TRUSTED_CA_FILE：etcd集群各节点相互认证使用的CA根证书文件全路径
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
# 192.168.56.105节点配置
cat << EOF > /etc/etcd/etcd.conf
ETCD_NAME=etcd1
ETCD_DATA_DIR=/etc/etcd/data

ETCD_CERT_FILE=/etc/etcd/pki/etcd_server.crt
ETCD_KEY_FILE=/etc/etcd/pki/etcd_server.key
ETCD_TRUSTED_CA_FILE=/etc/kubernetes/pki/ca.crt
ETCD_CLIENT_CERT_AUTH=true
ETCD_LISTEN_CLIENT_URLS=https://192.168.56.105:2379
ETCD_ADVERTISE_CLIENT_URLS=https://192.168.56.105:2379

ETCD_PEER_CERT_FILE=/etc/etcd/pki/etcd_server.crt
ETCD_PEER_KEY_FILE=/etc/etcd/pki/etcd_server.key
ETCD_PEER_TRUSTED_CA_FILE=/etc/kubernetes/pki/ca.crt
ETCD_LISTEN_PEER_URLS=https://192.168.56.105:2380
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://192.168.56.105:2380

ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
ETCD_INITIAL_CLUSTER="etcd1=https://192.168.56.105:2380,etcd2=https://192.168.56.106:2380,etcd3=https://192.168.56.107:2380"
ETCD_INITIAL_CLUSTER_STATE=new
EOF


# 192.168.56.106节点配置
cat << EOF > /etc/etcd/etcd.conf
ETCD_NAME=etcd2
ETCD_DATA_DIR=/etc/etcd/data

ETCD_CERT_FILE=/etc/etcd/pki/etcd_server.crt
ETCD_KEY_FILE=/etc/etcd/pki/etcd_server.key
ETCD_TRUSTED_CA_FILE=/etc/kubernetes/pki/ca.crt
ETCD_CLIENT_CERT_AUTH=true
ETCD_LISTEN_CLIENT_URLS=https://192.168.56.106:2379
ETCD_ADVERTISE_CLIENT_URLS=https://192.168.56.106:2379

ETCD_PEER_CERT_FILE=/etc/etcd/pki/etcd_server.crt
ETCD_PEER_KEY_FILE=/etc/etcd/pki/etcd_server.key
ETCD_PEER_TRUSTED_CA_FILE=/etc/kubernetes/pki/ca.crt
ETCD_LISTEN_PEER_URLS=https://192.168.56.106:2380
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://192.168.56.106:2380

ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
ETCD_INITIAL_CLUSTER="etcd1=https://192.168.56.105:2380,etcd2=https://192.168.56.106:2380,etcd3=https://192.168.56.107:2380"
ETCD_INITIAL_CLUSTER_STATE=new
EOF


# 192.168.56.107节点配置
cat << EOF > /etc/etcd/etcd.conf
ETCD_NAME=etcd3
ETCD_DATA_DIR=/etc/etcd/data

ETCD_CERT_FILE=/etc/etcd/pki/etcd_server.crt
ETCD_KEY_FILE=/etc/etcd/pki/etcd_server.key
ETCD_TRUSTED_CA_FILE=/etc/kubernetes/pki/ca.crt
ETCD_CLIENT_CERT_AUTH=true
ETCD_LISTEN_CLIENT_URLS=https://192.168.56.107:2379
ETCD_ADVERTISE_CLIENT_URLS=https://192.168.56.107:2379

ETCD_PEER_CERT_FILE=/etc/etcd/pki/etcd_server.crt
ETCD_PEER_KEY_FILE=/etc/etcd/pki/etcd_server.key
ETCD_PEER_TRUSTED_CA_FILE=/etc/kubernetes/pki/ca.crt
ETCD_LISTEN_PEER_URLS=https://192.168.56.107:2380
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://192.168.56.107:2380

ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
ETCD_INITIAL_CLUSTER="etcd1=https://192.168.56.105:2380,etcd2=https://192.168.56.106:2380,etcd3=https://192.168.56.107:2380"
ETCD_INITIAL_CLUSTER_STATE=new
EOF

创建系统服务文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# 所有节点执行
cat << EOF > /usr/lib/systemd/system/etcd.service
[Unit]
Description=etcd key-value store
Documentation=https://github.com/etcd-io/etcd
After=network.target

[Service]
EnvironmentFile=/etc/etcd/etcd.conf
ExecStart=/usr/bin/etcd
Restart=always

[Install]
WantedBy=multi-user.target
EOF

启动etcd服务

1
2
3
4
# 所有节点执行
systemctl daemon-reload
systemctl start etcd
systemctl enable etcd

五、etcd服务验证

1
2
3
4
5
6
# 任意节点执行
etcdctl --cacert=/etc/kubernetes/pki/ca.crt \
--cert=/etc/etcd/pki/etcd_client.crt \
--key=/etc/etcd/pki/etcd_client.key \
--endpoints=https://192.168.56.105:2379,https://192.168.56.106:2379,https://192.168.56.107:2379 \
endpoint health

结果截图

(•̀ᴗ•́)و ̑̑

Share