大学生做家教网站,WordPress设置模块间距,wordpress 清理缓存,企业咨询管理公司起名一、前言
在云原生技术席卷全球的今天#xff0c;Kubernetes#xff08;K8s#xff09;已成为容器编排领域的事实标准。当大家都习惯了kubeadm、kubeasz等自动化工具一键部署的便利时#xff0c;选择通过二进制方式手动搭建K8s集群更像是一场知其然亦知其所以然KubernetesK8s已成为容器编排领域的事实标准。当大家都习惯了kubeadm、kubeasz等自动化工具一键部署的便利时选择通过二进制方式手动搭建K8s集群更像是一场知其然亦知其所以然的深度修行。这种方式将带您穿透抽象层直面etcd的分布式存储机制、kube-apiserver的RESTful接口设计、kubelet与CRI的交互细节以及各个核心组件间的TLS双向认证体系。 本文将还原最原始的K8s部署逻辑从零开始构建一个符合生产环境要求的高可用集群。不同于封装工具对底层实现的隐藏二进制部署要求我们亲手配置每个组件的systemd服务单元精确编排CA证书链的签发流程甚至需要理解kube-proxy在不同模式下的iptables/ipvs规则生成机制。这种看似繁琐的过程恰恰是理解Kubernetes调度器如何通过watch机制协调集群状态、控制器如何通过reconcile循环维持系统稳定的最佳实践路径。 通过本系列教程您不仅将获得一个可横向扩展的集群架构包括多master节点负载均衡与etcd集群部署方案更重要的是建立起对K8s生态的立体认知。当面对证书过期、组件通信异常等生产环境常见故障时这份通过二进制部署积累的排障经验将成为您运维分布式系统的宝贵资产。让我们开始这场从源码到服务的探索之旅揭开Kubernetes神秘面纱背后的工程之美。 就在北京时间2025年4月23日上午K8S发布了最新版本K8S1.32.4版本于是我就使用二进制的方式部署出了一套完整的集群并且整理了最详细的笔记。 二、架构演示 三、基础环境
# 由于小编的学习环境实在是资源有限所以我把Etcd集群master集群和worker集群负载均衡高可用全部都放在了三个节点上面如果环境资源允许的话大家是可以解耦的因为这些服务不存在端口冲突的情况于是我就放在了一起部署使用。
操作系统主机名IP地址系统配置Ubuntu22.04-LTSnode-exporter4110.0.0.41/244Core/8G/100GBUbuntu22.04-LTSnode-exporter4210.0.0.42/244Core/8G/100GBUbuntu22.04-LTSnode-exporter4310.0.0.43/244Core/8G/100GB
四、开始部署 1.部署 Etcd 高可用集群
准备 Etcd 程序包 下载 Etcd 软件 wget https://github.com/etcd-io/etcd/releases/download/v3.5.21/etcd-v3.5.21-linux-amd64.tar.gz 解压 Etcd 的二进制程序包到 PATH 环境变量路 tar -xf etcd-v3.5.21-linux-amd64.tar.gz -C /usr/local/bin etcd-v3.5.21-linux-amd64/etcd{,ctl} --strip-components1 验证 etcdctl 版本 etcdctl version 将软件包下发到所有节点 使用 scp 命令将 /usr/local/bin 目录下的 etcd 和 etcdctl 文件复制到其他节点。
准备 Etcd 的证书文件 安装 cfssl 证书管理工具 wget http://192.168.16.253/Resources/Prometheus/softwares/Etcd/oldboyedu-cfssl-v1.6.5.zip
unzip oldboyedu-cfssl-v1.6.5.zip
apt install rename
rename -v s/_1.6.5_linux_amd64//g cfssl*
chmod x /usr/local/bin/cfssl* 创建证书存储目录 mkdir -pv /caofacan/certs/etcd cd /caofacan/certs/etcd/ 生成证书的 CSR 文件证书签发请求文件 cat etcd-ca-csr.json EOF
{
CN: etcd,
key: {algo: rsa,size: 2048
},
names: [{C: CN,ST: Beijing,L: Beijing,O: etcd,OU: Etcd Security}
],
ca: {expiry: 876000h
}
}
EOF 生成 Etcd CA 证书和 CA 证书的 key cfssl gencert -initca etcd-ca-csr.json | cfssljson -bare /caofacan/certs/etcd/etcd-ca 生成 Etcd 证书的有效期配置文件 cat ca-config.json EOF
{signing: {default: {expiry: 876000h},profiles: {kubernetes: {usages: [signing,key encipherment,server auth,client auth],expiry: 876000h}}}
}
EOF 生成 apiserver 证书的 CSR 文件 cat apiserver-csr.json EOF
{CN: kube-apiserver,key: {algo: rsa,size: 2048},names: [{C: CN,ST: Beijing,L: Beijing,O: Kubernetes,OU: Kubernetes-manual}]
}
EOF 基于自建 ca 证书生成 apiServer 的证书文件 cfssl gencert \-ca/caofacan/certs/etcd/etcd-ca.pem \-ca-key/caofacan/certs/etcd/etcd-ca-key.pem \-configca-config.json \--hostname10.200.0.1,10.0.0.240,kubernetes,kubernetes.default,kubernetes.default.svc,caofacan.com,kubernetes.default.svc.caofacan.com,10.0.0.41,10.0.0.42,10.0.0.43 \--profilekubernetes \apiserver-csr.json | cfssljson -bare /caofacan/certs/etcd/etcd-server
创建 Etcd 集群各节点配置 node-exporter41 节点的配置文件 mkdir -pv /caofacan/softwares/etcd
cat /caofacan/softwares/etcd/etcd.config.yml EOF
name: node-exporter41
data-dir: /var/lib/etcd
listen-peer-urls: https://10.0.0.41:2380
listen-client-urls: https://10.0.0.41:2379,http://127.0.0.1:2379
initial-cluster: node-exporter41https://10.0.0.41:2380,node-exporter42https://10.0.0.42:2380,node-exporter43https://10.0.0.43:2380
client-transport-security:cert-file: /caofacan/certs/etcd/etcd-server.pemkey-file: /caofacan/certs/etcd/etcd-server-key.pemca-file: /caofacan/certs/etcd/etcd-ca.pemclient-cert-auth: true
peer-transport-security:cert-file: /caofacan/certs/etcd/etcd-server.pemkey-file: /caofacan/certs/etcd/etcd-server-key.pemca-file: /caofacan/certs/etcd/etcd-ca.pempeer-client-cert-auth: true
EOF node-exporter42 和 node-exporter43 节点的配置文件类似 node-exporter41需修改对应 IP 和节点名称。
配置 Etcd 启动脚本
在每个节点上创建 /usr/lib/systemd/system/etcd.service 文件 [Unit]
DescriptionJason Yins Etcd Service
Documentationhttps://coreos.com/etcd/docs/latest/
Afternetwork.target[Service]
ExecStart/usr/local/bin/etcd --config-file/caofacan/softwares/etcd/etcd.config.yml
Restarton-failure
RestartSec10
LimitNOFILE65535[Install]
WantedBymulti-user.target
启动 Etcd 集群
systemctl daemon-reload systemctl enable --now etcd
systemctl status etcd
查看 Etcd 集群状态
etcdctl --endpoints10.0.0.41:2379,10.0.0.42:2379,10.0.0.43:2379 --cacert/caofacan/certs/etcd/etcd-ca.pem --cert/caofacan/certs/etcd/etcd-server.pem --key/caofacan/certs/etcd/etcd-server-key.pem endpoint status --write-outtable
输出示例
------------------------------------------------------------------------------------------------------------------------------
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
------------------------------------------------------------------------------------------------------------------------------
| 10.0.0.41:2379 | 9378902f41df91e9 | 3.5.21 | 4.9 MB | true | false | 30 | 58023 | 58023 | |
| 10.0.0.42:2379 | 18f972748ec1bd96 | 3.5.21 | 5.0 MB | false | false | 30 | 58023 | 58023 | |
| 10.0.0.43:2379 | a3dfd2d37c461ee9 | 3.5.21 | 4.9 MB | false | false | 30 | 58023 | 58023 | |
------------------------------------------------------------------------------------------------------------------------------
2.K8S 集群各主机环境准备
下载并解压 K8S 程序包
wget https://dl.k8s.io/v1.32.4/kubernetes-server-linux-amd64.tar.gz
tar xf kubernetes-server-linux-amd64-v1.32.4.tar.gz --strip-components3 -C /usr/local/bin kubernetes/server/bin/kube{let,ctl,-apiserver,-controller-manager,-scheduler,-proxy}
查看 kubelet 版本
kubelet --version
分发软件包到其他节点
for i in ls -1 /usr/local/bin/kube*; do data_rsync.sh $i; done
所有节点安装常用软件包
apt -y install bind9-utils expect rsync jq psmisc net-tools lvm2 vim unzip rename tree
node-exporter41 节点免密钥登录集群并同步数据 配置免密码登录其他节点 cat password_free_login.sh EOF
#!/bin/bash
# auther: Jason Yin# 创建密钥对
ssh-keygen -t rsa -P -f /root/.ssh/id_rsa -q# 声明你服务器密码建议所有节点的密码均一致否则该脚本需要再次进行优化
export mypasswd1# 定义主机列表
k8s_host_list(node-exporter41 node-exporter42 node-exporter43)# 配置免密登录利用expect工具免交互输入
for i in ${k8s_host_list[]};do
expect -c
spawn ssh-copy-id -i /root/.ssh/id_rsa.pub root$iexpect {*yes/no* {send yes\r; exp_continue}*password* {send $mypasswd\r; exp_continue}}
done
EOF
bash password_free_login.sh 编写同步脚本 cat /usr/local/sbin/data_rsync.sh EOF
#!/bin/bashif [ $# -lt 1 ];thenecho Usage: $0 /path/to/file(绝对路径) [mode: m|w]exit
fiif [ ! -e $1 ];thenecho [ $1 ] dir or file not find!exit
fifullpathdirname $1basenamebasename $1cd $fullpathcase $2 inWORKER_NODE|w)K8S_NODE(node-exporter42 node-exporter43);;MASTER_NODE|m)K8S_NODE(node-exporter42 node-exporter43);;*)K8S_NODE(node-exporter42 node-exporter43);;
esacfor host in ${K8S_NODE[]};dotput setaf 2echo rsyncing ${host}: $basename tput setaf 7rsync -az $basename whoami${host}:$fullpathif [ $? -eq 0 ];thenecho 命令执行成功!fi
done
EOF
chmod x /usr/local/sbin/data_rsync.sh
data_rsync.sh /etc/hosts
所有节点 Linux 基础环境优化
systemctl disable --now NetworkManager ufw
swapoff -a sysctl -w vm.swappiness0
sed -ri /^[^#]*swap/s^# /etc/fstab
free -h
ln -svf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
cat /etc/security/limits.conf EOF
* soft nofile 655360
* hard nofile 131072
* soft nproc 655350
* hard nproc 655350
* soft memlock unlimited
* hard memlock unlimited
EOF
sed -i s#UseDNS yesUseDNS nog /etc/ssh/sshd_config
sed -i s^GSSAPIAuthentication yesGSSAPIAuthentication nog /etc/ssh/sshd_config
cat /etc/sysctl.d/k8s.conf EOF
net.ipv4.ip_forward 1
net.bridge.bridge-nf-call-iptables 1
net.bridge.bridge-nf-call-ip6tables 1
net.ipv6.conf.all.disable_ipv6 1
fs.may_detach_mounts 1
vm.overcommit_memory1
vm.panic_on_oom0
fs.inotify.max_user_watches89100
fs.file-max52706963
fs.nr_open52706963
net.netfilter.nf_conntrack_max2310720
net.ipv4.tcp_keepalive_time 600
net.ipv4.tcp_keepalive_probes 3
net.ipv4.tcp_keepalive_intvl 15
net.ipv4.tcp_max_tw_buckets 36000
net.ipv4.tcp_tw_reuse 1
net.ipv4.tcp_max_orphans 327680
net.ipv4.tcp_orphan_retries 3
net.ipv4.tcp_syncookies 1
net.ipv4.tcp_max_syn_backlog 16384
net.ipv4.ip_conntrack_max 65536
net.ipv4.tcp_max_syn_backlog 16384
net.ipv4.tcp_timestamps 0
net.core.somaxconn 16384
EOF
sysctl --system
cat EOF ~/.bashrc
PS1[\[\e[34;1m\]\u\[\e[0m\]\[\e[32;1m\]\H\[\e[0m\]\[\e[31;1m\] \W\[\e[0m\]]#
EOF
source ~/.bashrc
所有节点安装 ipvsadm 以实现 kube-proxy 的负载均衡
apt -y install ipvsadm ipset sysstat conntrack
cat /etc/modules-load.d/ipvs.conf EOF
ip_vs
ip_vs_lc
ip_vs_wlc
ip_vs_rr
ip_vs_wrr
ip_vs_lblc
ip_vs_lblcr
ip_vs_dh
ip_vs_sh
ip_vs_fo
ip_vs_nq
ip_vs_sed
ip_vs_ftp
ip_vs_sh
nf_conntrack
br_netfilter
ip_tables
ip_set
xt_set
ipt_set
ipt_rpfilter
ipt_REJECT
ipip
EOF
验证模块是否加载成功
reboot
lsmod | grep --colorauto -e ip_vs -e nf_conntrack -e br_netfilter
部署 Containerd 运行时
# 这里我自己写了一个自动部署containerd脚本
tar xf oldboyedu-autoinstall-containerd-v1.6.36.tar.gz
./install-containerd.sh i
检查 Containerd 版本
ctr version 3.生成 K8S 组件相关证书
生成 K8S 组件相关证书
创建证书的 CSR 文件
mkdir -pv /caofacan/certs/pki cd /caofacan/certs/pki
cat k8s-ca-csr.json EOF
{
CN: kubernetes,
key: {algo: rsa,size: 2048
},
names: [{C: CN,ST: Beijing,L: Beijing,O: Kubernetes,OU: Kubernetes-manual}
],
ca: {expiry: 876000h
}
}
EOF
生成 K8S 证书
mkdir -pv /caofacan/certs/kubernetes/
cfssl gencert -initca k8s-ca-csr.json | cfssljson -bare /caofacan/certs/kubernetes/k8s-ca
生成 K8S 证书的有效期为 100 年
cat k8s-ca-config.json EOF
{signing: {default: {expiry: 876000h},profiles: {kubernetes: {usages: [signing,key encipherment,server auth,client auth],expiry: 876000h}}}
}
EOF
生成 apiserver 证书的 CSR 文件
cat apiserver-csr.json EOF
{CN: kube-apiserver,key: {algo: rsa,size: 2048},names: [{C: CN,ST: Beijing,L: Beijing,O: Kubernetes,OU: Kubernetes-manual}]
}
EOF
基于自建 ca 证书生成 apiServer 的证书文件
cfssl gencert \
-ca/caofacan/certs/kubernetes/k8s-ca.pem \
-ca-key/caofacan/certs/kubernetes/k8s-ca-key.pem \
-configk8s-ca-config.json \
--hostname10.200.0.1,10.0.0.240,kubernetes,kubernetes.default,kubernetes.default.svc,caofacan.com,kubernetes.default.svc.caofacan.com,10.0.0.41,10.0.0.42,10.0.0.43 \
--profilekubernetes \
apiserver-csr.json | cfssljson -bare /caofacan/certs/kubernetes/apiserver
生成聚合证书的用于自建 ca 的 CSR 文件
cat front-proxy-ca-csr.json EOF
{CN: kubernetes,key: {algo: rsa,size: 2048}
}
EOF
生成聚合证书的自建 ca 证书
cfssl gencert -initca front-proxy-ca-csr.json | cfssljson -bare /caofacan/certs/kubernetes/front-proxy-ca
生成聚合证书的用于客户端的 CSR 文件
cat front-proxy-client-csr.json EOF
{CN: front-proxy-client,key: {algo: rsa,size: 2048}
}
EOF
基于聚合证书的自建 ca 证书签发聚合证书的客户端证书
cfssl gencert \
-ca/caofacan/certs/kubernetes/front-proxy-ca.pem \
-ca-key/caofacan/certs/kubernetes/front-proxy-ca-key.pem \
-configk8s-ca-config.json \
-profilekubernetes \
front-proxy-client-csr.json | cfssljson -bare /caofacan/certs/kubernetes/front-proxy-client
4.生成 controller-manager 证书及 kubeconfig 文件 生成 controller-manager 的 CSR 文件 cat controller-manager-csr.json EOF
{CN: system:kube-controller-manager,key: {algo: rsa,size: 2048},names: [{C: CN,ST: Beijing,L: Beijing,O: system:kube-controller-manager,OU: Kubernetes-manual}]
}
EOF 生成 controller-manager 证书文件 cfssl gencert \-ca/caofacan/certs/kubernetes/k8s-ca.pem \-ca-key/caofacan/certs/kubernetes/k8s-ca-key.pem \-configk8s-ca-config.json \-profilekubernetes \controller-manager-csr.json | cfssljson -bare /caofacan/certs/kubernetes/controller-manager 创建 kubeconfig 目录 mkdir -pv /caofacan/certs/kubeconfig 设置集群 kubectl config set-cluster yinzhengjie-k8s \--certificate-authority/caofacan/certs/kubernetes/k8s-ca.pem \--embed-certstrue \--serverhttps://10.0.0.240:8443 \--kubeconfig/caofacan/certs/kubeconfig/kube-controller-manager.kubeconfig 设置用户项 kubectl config set-credentials system:kube-controller-manager \--client-certificate/caofacan/certs/kubernetes/controller-manager.pem \--client-key/caofacan/certs/kubernetes/controller-manager-key.pem \--embed-certstrue \--kubeconfig/caofacan/certs/kubeconfig/kube-controller-manager.kubeconfig 设置上下文环境 kubectl config set-context system:kube-controller-managerkubernetes \--clusteryinzhengjie-k8s \--usersystem:kube-controller-manager \--kubeconfig/caofacan/certs/kubeconfig/kube-controller-manager.kubeconfig 使用默认上下文 kubectl config use-context system:kube-controller-managerkubernetes \--kubeconfig/caofacan/certs/kubeconfig/kube-controller-manager.kubeconfig
5.生成 scheduler 证书及 kubeconfig 文件 生成 scheduler 的 CSR 文件 cat scheduler-csr.json EOF
{CN: system:kube-scheduler,key: {algo: rsa,size: 2048},names: [{C: CN,ST: Beijing,L: Beijing,O: system:kube-scheduler,OU: Kubernetes-manual}]
}
EOF 生成 scheduler 证书文件 cfssl gencert \-ca/caofacan/certs/kubernetes/k8s-ca.pem \-ca-key/caofacan/certs/kubernetes/k8s-ca-key.pem \-configk8s-ca-config.json \-profilekubernetes \scheduler-csr.json | cfssljson -bare /caofacan/certs/kubernetes/scheduler 设置集群 kubectl config set-cluster yinzhengjie-k8s \--certificate-authority/caofacan/certs/kubernetes/k8s-ca.pem \--embed-certstrue \--serverhttps://10.0.0.240:8443 \--kubeconfig/caofacan/certs/kubeconfig/kube-scheduler.kubeconfig 设置用户项 kubectl config set-credentials system:kube-scheduler \--client-certificate/caofacan/certs/kubernetes/scheduler.pem \--client-key/caofacan/certs/kubernetes/scheduler-key.pem \--embed-certstrue \--kubeconfig/caofacan/certs/kubeconfig/kube-scheduler.kubeconfig 设置上下文环境 kubectl config set-context system:kube-schedulerkubernetes \--clusteryinzhengjie-k8s \--usersystem:kube-scheduler \--kubeconfig/caofacan/certs/kubeconfig/kube-scheduler.kubeconfig 使用默认上下文 kubectl config use-context system:kube-schedulerkubernetes \--kubeconfig/caofacan/certs/kubeconfig/kube-scheduler.kubeconfig
6.配置 K8S 集群管理员证书及 kubeconfig 文件 生成管理员的 CSR 文件 cat admin-csr.json EOF
{CN: admin,key: {algo: rsa,size: 2048},names: [{C: CN,ST: Beijing,L: Beijing,O: system:masters,OU: Kubernetes-manual}]
}
EOF 生成 K8S 集群管理员证书 cfssl gencert \-ca/caofacan/certs/kubernetes/k8s-ca.pem \-ca-key/caofacan/certs/kubernetes/k8s-ca-key.pem \-configk8s-ca-config.json \-profilekubernetes \admin-csr.json | cfssljson -bare /caofacan/certs/kubernetes/admin 设置集群 kubectl config set-cluster yinzhengjie-k8s \--certificate-authority/caofacan/certs/kubernetes/k8s-ca.pem \--embed-certstrue \--serverhttps://10.0.0.240:8443 \--kubeconfig/caofacan/certs/kubeconfig/kube-admin.kubeconfig 设置用户项 kubectl config set-credentials kube-admin \--client-certificate/caofacan/certs/kubernetes/admin.pem \--client-key/caofacan/certs/kubernetes/admin-key.pem \--embed-certstrue \--kubeconfig/caofacan/certs/kubeconfig/kube-admin.kubeconfig 设置上下文环境 kubectl config set-context kube-adminkubernetes \--clusteryinzhengjie-k8s \--userkube-admin \--kubeconfig/caofacan/certs/kubeconfig/kube-admin.kubeconfig 使用默认上下文 kubectl config use-context kube-adminkubernetes \--kubeconfig/caofacan/certs/kubeconfig/kube-admin.kubeconfig
7.创建 ServiceAccount
openssl genrsa -out /caofacan/certs/kubernetes/sa.key 2048
openssl rsa -in /caofacan/certs/kubernetes/sa.key -pubout -out /caofacan/certs/kubernetes/sa.pub 将 K8S 组件证书拷贝到其他两个 master 节点
data_rsync.sh /caofacan/certs/kubeconfig/
data_rsync.sh /caofacan/certs/kubernetes/
8.高可用组件 haproxy keepalived 安装及验证
所有 master 节点安装高可用组件
apt-get -y install keepalived haproxy
配置 haproxy 备份配置文件 cp /etc/haproxy/haproxy.cfg{,date %F} 所有节点的配置文件内容相同 cat /etc/haproxy/haproxy.cfg EOF
globalmaxconn 2000ulimit-n 16384log 127.0.0.1 local0 errstats timeout 30sdefaultslog globalmode httpoption httplogtimeout connect 5000timeout client 50000timeout server 50000timeout http-request 15stimeout http-keep-alive 15sfrontend monitor-haproxybind *:9999mode httpoption httplogmonitor-uri /ruokfrontend yinzhengjie-k8sbind 0.0.0.0:8443bind 127.0.0.1:8443mode tcpoption tcplogtcp-request inspect-delay 5sdefault_backend yinzhengjie-k8sbackend yinzhengjie-k8smode tcpoption tcplogoption tcp-checkbalance roundrobindefault-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100server node-exporter41 10.0.0.41:6443 checkserver node-exporter42 10.0.0.42:6443 checkserver node-exporter43 10.0.0.43:6443 check
EOF
配置 keepalived node-exporter41 节点创建配置文件 cat /etc/keepalived/keepalived.conf EOF
! Configuration File for keepalived
global_defs {router_id 10.0.0.41
}
vrrp_script chk_nginx {script /etc/keepalived/check_port.sh 8443interval 2weight -20
}
vrrp_instance VI_1 {state MASTERinterface eth0virtual_router_id 251priority 100advert_int 1mcast_src_ip 10.0.0.41nopreemptauthentication {auth_type PASSauth_pass yinzhengjie_k8s}track_script {chk_nginx}virtual_ipaddress {10.0.0.240}
}
EOF node-exporter42 和 node-exporter43 节点创建配置文件类似 node-exporter41需修改对应 IP。 所有 keepalived 节点创建健康检查脚本 cat /etc/keepalived/check_port.sh EOF
#!/bin/bashCHK_PORT$1
if [ -n $CHK_PORT ];thenPORT_PROCESSss -lt|grep $CHK_PORT|wc -lif [ $PORT_PROCESS -eq 0 ];thenecho Port $CHK_PORT Is Not Used,End.systemctl stop keepalivedfi
elseecho Check Port Cant Be Empty!
fi
EOF
chmod x /etc/keepalived/check_port.sh
验证 haproxy 服务 所有节点启动 haproxy 服务 systemctl enable --now haproxy
systemctl restart haproxy
systemctl status haproxy
ss -ntl | egrep 8443|9999 基于 webUI 进行验证 curl http://10.0.0.41:9999/ruok
curl http://10.0.0.42:9999/ruok
curl http://10.0.0.43:9999/ruok
启动 keepalived 服务并验证 所有节点启动 keepalived 服务 systemctl daemon-reload
systemctl enable --now keepalived
systemctl status keepalived 验证服务是否正常 ip a 基于 telnet 验证 haproxy 是否正常 telnet 10.0.0.240 8443
ping 10.0.0.240 -c 3 9.部署 ApiServer 组件
node-exporter41 节点启动 ApiServer 创建 node-exporter41 节点的配置文件 cat /usr/lib/systemd/system/kube-apiserver.service EOF
[Unit]
DescriptionJason Yins Kubernetes API Server
Documentationhttps://github.com/kubernetes/kubernetes
Afternetwork.target[Service]
ExecStart/usr/local/bin/kube-apiserver \--v2 \--bind-address0.0.0.0 \--secure-port6443 \--allow_privilegedtrue \--advertise-address10.0.0.41 \--service-cluster-ip-range10.200.0.0/16 \--service-node-port-range3000-50000 \--etcd-servershttps://10.0.0.41:2379,https://10.0.0.42:2379,https://10.0.0.43:2379 \--etcd-cafile/caofacan/certs/etcd/etcd-ca.pem \--etcd-certfile/caofacan/certs/etcd/etcd-server.pem \--etcd-keyfile/caofacan/certs/etcd/etcd-server-key.pem \--client-ca-file/caofacan/certs/kubernetes/k8s-ca.pem \--tls-cert-file/caofacan/certs/kubernetes/apiserver.pem \--tls-private-key-file/caofacan/certs/kubernetes/apiserver-key.pem \--kubelet-client-certificate/caofacan/certs/kubernetes/apiserver.pem \--kubelet-client-key/caofacan/certs/kubernetes/apiserver-key.pem \--service-account-key-file/caofacan/certs/kubernetes/sa.pub \--service-account-signing-key-file/caofacan/certs/kubernetes/sa.key \--service-account-issuerhttps://kubernetes.default.svc.caofacan.com \--kubelet-preferred-address-typesInternalIP,ExternalIP,Hostname \--enable-admission-pluginsNamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota \--authorization-modeNode,RBAC \--enable-bootstrap-token-authtrue \--requestheader-client-ca-file/caofacan/certs/kubernetes/front-proxy-ca.pem \--proxy-client-cert-file/caofacan/certs/kubernetes/front-proxy-client.pem \--proxy-client-key-file/caofacan/certs/kubernetes/front-proxy-client-key.pem \--requestheader-allowed-namesaggregator \--requestheader-group-headersX-Remote-Group \--requestheader-extra-headers-prefixX-Remote-Extra- \--requestheader-username-headersX-Remote-UserRestarton-failure
RestartSec10s
LimitNOFILE65535[Install]
WantedBymulti-user.target
EOF 启动服务 systemctl daemon-reload systemctl enable --now kube-apiserver
systemctl status kube-apiserver
ss -ntl | grep 6443
node-exporter42 和 node-exporter43 节点启动 ApiServer
类似 node-exporter41需修改对应 IP 和节点名称。
10.部署 ControllerManager 组件
所有节点创建配置文件
cat /usr/lib/systemd/system/kube-controller-manager.service EOF
[Unit]
DescriptionJason Yins Kubernetes Controller Manager
Documentationhttps://github.com/kubernetes/kubernetes
Afternetwork.target[Service]
ExecStart/usr/local/bin/kube-controller-manager \--v2 \--root-ca-file/caofacan/certs/kubernetes/k8s-ca.pem \--cluster-signing-cert-file/caofacan/certs/kubernetes/k8s-ca.pem \--cluster-signing-key-file/caofacan/certs/kubernetes/k8s-ca-key.pem \--service-account-private-key-file/caofacan/certs/kubernetes/sa.key \--kubeconfig/caofacan/certs/kubeconfig/kube-controller-manager.kubeconfig \--leader-electtrue \--use-service-account-credentialstrue \--node-monitor-grace-period40s \--node-monitor-period5s \--controllers*,bootstrapsigner,tokencleaner \--allocate-node-cidrstrue \--cluster-cidr10.100.0.0/16 \--requestheader-client-ca-file/caofacan/certs/kubernetes.front-proxy-ca.pem \--node-cidr-mask-size24Restartalways
RestartSec10s[Install]
WantedBymulti-user.target
EOF
启动 controller-manager 服务
systemctl daemon-reload
systemctl enable --now kube-controller-manager
systemctl status kube-controller-manager
ss -ntl | grep 10257
11.部署 Scheduler 组件
所有节点创建配置文件
cat /usr/lib/systemd/system/kube-scheduler.service EOF
[Unit]
DescriptionJason Yins Kubernetes Scheduler
Documentationhttps://github.com/kubernetes/kubernetes
Afternetwork.target[Service]
ExecStart/usr/local/bin/kube-scheduler \--v2 \--leader-electtrue \--kubeconfig/caofacan/certs/kubeconfig/kube-scheduler.kubeconfigRestartalways
RestartSec10s[Install]
WantedBymulti-user.target
EOF
启动 scheduler 服务
systemctl daemon-reload
systemctl enable --now kube-scheduler
systemctl status kube-scheduler
ss -ntl | grep 10259 12.创建 Bootstrapping 自动颁发 kubelet 证书配置及集群管理证书配置
创建 bootstrap-kubelet.kubeconfig 文件 设置集群 kubectl config set-cluster yinzhengjie-k8s \--certificate-authority/caofacan/certs/kubernetes/k8s-ca.pem \--embed-certstrue \--serverhttps://10.0.0.240:8443 \--kubeconfig/caofacan/certs/kubeconfig/bootstrap-kubelet.kubeconfig 创建用户 kubectl config set-credentials tls-bootstrap-token-user \--tokencaofacan.jasonyinzhengjie \--kubeconfig/caofacan/certs/kubeconfig/bootstrap-kubelet.kubeconfig 将集群和用户进行绑定 kubectl config set-context tls-bootstrap-token-userkubernetes \--clusteryinzhengjie-k8s \--usertls-bootstrap-token-user \--kubeconfig/caofacan/certs/kubeconfig/bootstrap-kubelet.kubeconfig 配置默认的上下文 kubectl config use-context tls-bootstrap-token-userkubernetes \--kubeconfig/caofacan/certs/kubeconfig/bootstrap-kubelet.kubeconfig 拷贝 kubelet 的 Kubeconfig 文件 data_rsync.sh /caofacan/certs/kubeconfig/bootstrap-kubelet.kubeconfig 13.所有 master 节点拷贝管理证书 所有 master 节点拷贝管理员的证书文件 mkdir -p /root/.kube
cp /caofacan/certs/kubeconfig/kube-admin.kubeconfig /root/.kube/config 查看 master 组件 kubectl get cs 查看集群状态 kubectl cluster-info dump
14.创建 bootstrap-secret 授权 创建 bootstrap-secret 文件用于授权 cat bootstrap-secret.yaml EOF
apiVersion: v1
kind: Secret
metadata:name: bootstrap-token-caofacannamespace: kube-system
type: bootstrap.kubernetes.io/token
stringData:description: The default bootstrap token generated by kubelet .token-id: caofacantoken-secret: jasonyinzhengjieusage-bootstrap-authentication: trueusage-bootstrap-signing: trueauth-extra-groups: system:bootstrappers:default-node-token,system:bootstrappers:worker,system:bootstrappers:ingress---apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: kubelet-bootstrap
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:node-bootstrapper
subjects:
- apiGroup: rbac.authorization.k8s.iokind: Groupname: system:bootstrappers:default-node-token---apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: node-autoapprove-bootstrap
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:certificates.k8s.io:certificatesigningrequests:nodeclient
subjects:
- apiGroup: rbac.authorization.k8s.iokind: Groupname: system:bootstrappers:default-node-token---apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: node-autoapprove-certificate-rotation
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:certificates.k8s.io:certificatesigningrequests:selfnodeclient
subjects:
- apiGroup: rbac.authorization.k8s.iokind: Groupname: system:nodes---apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:annotations:rbac.authorization.kubernetes.io/autoupdate: truelabels:kubernetes.io/bootstrapping: rbac-defaultsname: system:kube-apiserver-to-kubelet
rules:- apiGroups:- resources:- nodes/proxy- nodes/stats- nodes/log- nodes/spec- nodes/metricsverbs:- *---apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: system:kube-apiservernamespace:
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:kube-apiserver-to-kubelet
subjects:- apiGroup: rbac.authorization.k8s.iokind: Username: kube-apiserver
EOF 应用 bootstrap-secret 配置文件 kubectl apply -f bootstrap-secret.yaml
15.部署 worker 节点之 kubelet 启动实战
所有节点创建工作目录
mkdir -p /var/lib/kubelet /var/log/kubernetes /etc/systemd/system/kubelet.service.d /etc/kubernetes/manifests/
所有节点创建 kubelet 的配置文件
cat /etc/kubernetes/kubelet-conf.yml EOF
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
address: 0.0.0.0
port: 10250
readOnlyPort: 10255
authentication:anonymous:enabled: falsewebhook:cacheTTL: 2m0senabled: truex509:clientCAFile: /caofacan/certs/kubernetes/k8s-ca.pem
authorization:mode: Webhookwebhook:cacheAuthorizedTTL: 5m0scacheUnauthorizedTTL: 30s
cgroupDriver: systemd
cgroupsPerQOS: true
clusterDNS:
- 10.200.0.254
clusterDomain: caofacan.com
containerLogMaxFiles: 5
containerLogMaxSize: 10Mi
contentType: application/vnd.kubernetes.protobuf
cpuCFSQuota: true
cpuManagerPolicy: none
cpuManagerReconcilePeriod: 10s
enableControllerAttachDetach: true
enableDebuggingHandlers: true
enforceNodeAllocatable:
- pods
eventBurst: 10
eventRecordQPS: 5
evictionHard:imagefs.available: 15%memory.available: 100Minodefs.available: 10%nodefs.inodesFree: 5%
evictionPressureTransitionPeriod: 5m0s
failSwapOn: true
fileCheckFrequency: 20s
hairpinMode: promiscuous-bridge
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 20s
imageGCHighThresholdPercent: 85
imageGCLowThresholdPercent: 80
imageMinimumGCAge: 2m0s
iptablesDropBit: 15
iptablesMasqueradeBit: 14
kubeAPIBurst: 10
kubeAPIQPS: 5
makeIPTablesUtilChains: true
maxOpenFiles: 1000000
maxPods: 110
nodeStatusUpdateFrequency: 10s
oomScoreAdj: -999
podPidsLimit: -1
registryBurst: 10
registryPullQPS: 5
resolvConf: /etc/kubernetes/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 2m0s
serializeImagePulls: true
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 4h0m0s
syncFrequency: 1m0s
volumeStatsAggPeriod: 1m0s
EOF
所有节点配置 kubelet service
cat /usr/lib/systemd/system/kubelet.service EOF
[Unit]
DescriptionJasonYins Kubernetes Kubelet
Documentationhttps://github.com/kubernetes/kubernetes
Aftercontainerd.service
Requirescontainerd.service[Service]
ExecStart/usr/local/bin/kubelet
Restartalways
StartLimitInterval0
RestartSec10[Install]
WantedBymulti-user.target
EOF
所有节点配置 kubelet service 的配置文件
cat /etc/systemd/system/kubelet.service.d/10-kubelet.conf EOF
[Service]
EnvironmentKUBELET_KUBECONFIG_ARGS--bootstrap-kubeconfig/caofacan/certs/kubeconfig/bootstrap-kubelet.kubeconfig --kubeconfig/caofacan/certs/kubeconfig/kubelet.kubeconfig
EnvironmentKUBELET_CONFIG_ARGS--config/etc/kubernetes/kubelet-conf.yml
EnvironmentKUBELET_SYSTEM_ARGS--container-runtime-endpointunix:///run/containerd/containerd.sock
EnvironmentKUBELET_EXTRA_ARGS--node-labelsnode.kubernetes.io/node
ExecStart
ExecStart/usr/local/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_SYSTEM_ARGS $KUBELET_EXTRA_ARGS
EOF
启动所有节点 kubelet
systemctl daemon-reload
systemctl enable --now kubelet
systemctl status kubelet
在所有 master 节点上查看 nodes 信息
kubectl get nodes -o wide
查看 csr 用户客户端的证书请求
kubectl get csr
16.部署 worker 节点之 kube-proxy 服务
生成 kube-proxy 的 csr 文件
cat kube-proxy-csr.json EOF
{
CN: system:kube-proxy,
key: {algo: rsa,size: 2048
},
names: [{C: CN,ST: Beijing,L: Beijing,O: system:kube-proxy,OU: Kubernetes-manual}
]
}
EOF
创建 kube-proxy 需要的证书文件
cfssl gencert \
-ca/caofacan/certs/kubernetes/k8s-ca.pem \
-ca-key/caofacan/certs/kubernetes/k8s-ca-key.pem \
-configk8s-ca-config.json \
-profilekubernetes \
kube-proxy-csr.json | cfssljson -bare /caofacan/certs/kubernetes/kube-proxy
生成 Kubeconfig 文件 设置集群 kubectl config set-cluster yinzhengjie-k8s \--certificate-authority/caofacan/certs/kubernetes/k8s-ca.pem \--embed-certstrue \--serverhttps://10.0.0.240:8443 \--kubeconfig/caofacan/certs/kubeconfig/kube-proxy.kubeconfig 设置用户项 kubectl config set-credentials system:kube-proxy \--client-certificate/caofacan/certs/kubernetes/kube-proxy.pem \--client-key/caofacan/certs/kubernetes/kube-proxy-key.pem \--embed-certstrue \--kubeconfig/caofacan/certs/kubeconfig/kube-proxy.kubeconfig 设置上下文环境 kubectl config set-context kube-proxykubernetes \--clusteryinzhengjie-k8s \--usersystem:kube-proxy \--kubeconfig/caofacan/certs/kubeconfig/kube-proxy.kubeconfig 使用默认上下文 kubectl config use-context kube-proxykubernetes \--kubeconfig/caofacan/certs/kubeconfig/kube-proxy.kubeconfig
将 Kubeconfig 文件同步到所有的 worker 节点 data_rsync.sh /caofacan/certs/kubeconfig/kube-proxy.kubeconfig
所有节点创建 kube-proxy.conf 配置文件 cat /etc/kubernetes/kube-proxy.yml EOF
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
bindAddress: 0.0.0.0
metricsBindAddress: 127.0.0.1:10249
clientConnection:acceptConnection: burst: 10contentType: application/vnd.kubernetes.protobufkubeconfig: /caofacan/certs/kubeconfig/kube-proxy.kubeconfigqps: 5
clusterCIDR: 10.100.0.0/16
configSyncPeriod: 15m0s
conntrack:max: nullmaxPerCore: 32768min: 131072tcpCloseWaitTimeout: 1h0m0stcpEstablishedTimeout: 24h0m0s
enableProfiling: false
healthzBindAddress: 0.0.0.0:10256
hostnameOverride:
iptables:masqueradeAll: falsemasqueradeBit: 14minSyncPeriod: 0s
ipvs:masqueradeAll: trueminSyncPeriod: 5sscheduler: rrsyncPeriod: 30s
mode: ipvs
nodeProtAddress: null
oomScoreAdj: -999
portRange:
udpIdelTimeout: 250ms
EOF
所有节点使用 systemd 管理 kube-proxy cat /usr/lib/systemd/system/kube-proxy.service EOF
[Unit]
DescriptionJason Yins Kubernetes Proxy
Afternetwork.target[Service]
ExecStart/usr/local/bin/kube-proxy \--config/etc/kubernetes/kube-proxy.yml \--v2
Restarton-failure
LimitNOFILE65536[Install]
WantedBymulti-user.target
EOF
所有节点启动 kube-proxy systemctl daemon-reload systemctl enable --now kube-proxy
systemctl status kube-proxy
ss -ntl |egrep 10256|10249 17.附加组件部署CNI之flannel
参考链接:https://github.com/flannel-io/flannel?tabreadme-ov-file#deploying-flannel-manually1.下载资源清单 2.修改Flannel的资源清单
[rootnode-exporter41 ~]# grep 16 kube-flannel.yml Network: 10.244.0.0/16,
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# sed -i /16/s#244#100# kube-flannel.yml
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# grep 16 kube-flannel.yml Network: 10.100.0.0/16,
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# 3.所有节点安装CNI通用插件
wget https://github.com/containernetworking/plugins/releases/download/v1.6.2/cni-plugins-linux-amd64-v1.6.2.tgztar xf cni-plugins-linux-amd64-v1.6.2.tgz -C /opt/cni/bin/[rootnode-exporter41 ~]# ll /opt/cni/bin/
total 89780
drwxr-xr-x 2 root root 4096 Jan 7 00:12 ./
drwxr-xr-x 3 root root 4096 Apr 23 14:43 ../
-rwxr-xr-x 1 root root 4655178 Jan 7 00:12 bandwidth*
-rwxr-xr-x 1 root root 5287212 Jan 7 00:12 bridge*
-rwxr-xr-x 1 root root 12762814 Jan 7 00:12 dhcp*
-rwxr-xr-x 1 root root 4847854 Jan 7 00:12 dummy*
-rwxr-xr-x 1 root root 5315134 Jan 7 00:12 firewall*
-rwxr-xr-x 1 root root 4792010 Jan 7 00:12 host-device*
-rwxr-xr-x 1 root root 4060355 Jan 7 00:12 host-local*
-rwxr-xr-x 1 root root 4870719 Jan 7 00:12 ipvlan*
-rw-r--r-- 1 root root 11357 Jan 7 00:12 LICENSE
-rwxr-xr-x 1 root root 4114939 Jan 7 00:12 loopback*
-rwxr-xr-x 1 root root 4903324 Jan 7 00:12 macvlan*
-rwxr-xr-x 1 root root 4713429 Jan 7 00:12 portmap*
-rwxr-xr-x 1 root root 5076613 Jan 7 00:12 ptp*
-rw-r--r-- 1 root root 2343 Jan 7 00:12 README.md
-rwxr-xr-x 1 root root 4333422 Jan 7 00:12 sbr*
-rwxr-xr-x 1 root root 3651755 Jan 7 00:12 static*
-rwxr-xr-x 1 root root 4928874 Jan 7 00:12 tap*
-rwxr-xr-x 1 root root 4208424 Jan 7 00:12 tuning*
-rwxr-xr-x 1 root root 4868252 Jan 7 00:12 vlan*
-rwxr-xr-x 1 root root 4488658 Jan 7 00:12 vrf*
[rootnode-exporter41 ~]# [rootnode-exporter42 ~]# ll /opt/cni/bin/
total 89780
drwxr-xr-x 2 root root 4096 Apr 23 14:56 ./
drwxr-xr-x 3 root root 4096 Apr 23 14:43 ../
-rwxr-xr-x 1 root root 4655178 Jan 7 00:12 bandwidth*
-rwxr-xr-x 1 root root 5287212 Jan 7 00:12 bridge*
-rwxr-xr-x 1 root root 12762814 Jan 7 00:12 dhcp*
-rwxr-xr-x 1 root root 4847854 Jan 7 00:12 dummy*
-rwxr-xr-x 1 root root 5315134 Jan 7 00:12 firewall*
-rwxr-xr-x 1 root root 4792010 Jan 7 00:12 host-device*
-rwxr-xr-x 1 root root 4060355 Jan 7 00:12 host-local*
-rwxr-xr-x 1 root root 4870719 Jan 7 00:12 ipvlan*
-rw-r--r-- 1 root root 11357 Jan 7 00:12 LICENSE
-rwxr-xr-x 1 root root 4114939 Jan 7 00:12 loopback*
-rwxr-xr-x 1 root root 4903324 Jan 7 00:12 macvlan*
-rwxr-xr-x 1 root root 4713429 Jan 7 00:12 portmap*
-rwxr-xr-x 1 root root 5076613 Jan 7 00:12 ptp*
-rw-r--r-- 1 root root 2343 Jan 7 00:12 README.md
-rwxr-xr-x 1 root root 4333422 Jan 7 00:12 sbr*
-rwxr-xr-x 1 root root 3651755 Jan 7 00:12 static*
-rwxr-xr-x 1 root root 4928874 Jan 7 00:12 tap*
-rwxr-xr-x 1 root root 4208424 Jan 7 00:12 tuning*
-rwxr-xr-x 1 root root 4868252 Jan 7 00:12 vlan*
-rwxr-xr-x 1 root root 4488658 Jan 7 00:12 vrf*
[rootnode-exporter42 ~]#
[rootnode-exporter42 ~]# [rootnode-exporter43 ~]# ll /opt/cni/bin/
total 89780
drwxr-xr-x 2 root root 4096 Apr 23 14:56 ./
drwxr-xr-x 3 root root 4096 Apr 23 14:43 ../
-rwxr-xr-x 1 root root 4655178 Jan 7 00:12 bandwidth*
-rwxr-xr-x 1 root root 5287212 Jan 7 00:12 bridge*
-rwxr-xr-x 1 root root 12762814 Jan 7 00:12 dhcp*
-rwxr-xr-x 1 root root 4847854 Jan 7 00:12 dummy*
-rwxr-xr-x 1 root root 5315134 Jan 7 00:12 firewall*
-rwxr-xr-x 1 root root 4792010 Jan 7 00:12 host-device*
-rwxr-xr-x 1 root root 4060355 Jan 7 00:12 host-local*
-rwxr-xr-x 1 root root 4870719 Jan 7 00:12 ipvlan*
-rw-r--r-- 1 root root 11357 Jan 7 00:12 LICENSE
-rwxr-xr-x 1 root root 4114939 Jan 7 00:12 loopback*
-rwxr-xr-x 1 root root 4903324 Jan 7 00:12 macvlan*
-rwxr-xr-x 1 root root 4713429 Jan 7 00:12 portmap*
-rwxr-xr-x 1 root root 5076613 Jan 7 00:12 ptp*
-rw-r--r-- 1 root root 2343 Jan 7 00:12 README.md
-rwxr-xr-x 1 root root 4333422 Jan 7 00:12 sbr*
-rwxr-xr-x 1 root root 3651755 Jan 7 00:12 static*
-rwxr-xr-x 1 root root 4928874 Jan 7 00:12 tap*
-rwxr-xr-x 1 root root 4208424 Jan 7 00:12 tuning*
-rwxr-xr-x 1 root root 4868252 Jan 7 00:12 vlan*
-rwxr-xr-x 1 root root 4488658 Jan 7 00:12 vrf*
[rootnode-exporter43 ~]# 4.部署服务
[rootnode-exporter41 ~]# kubectl apply -f kube-flannel.yml
namespace/kube-flannel created
serviceaccount/flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
[rootnode-exporter41 ~]# 5.检查配置
[rootnode-exporter41 ~]# kubectl get pods -o wide -A
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-flannel kube-flannel-ds-8pt5n 1/1 Running 0 5s 10.0.0.41 node-exporter41 none none
kube-flannel kube-flannel-ds-gwhpb 1/1 Running 0 5s 10.0.0.43 node-exporter43 none none
kube-flannel kube-flannel-ds-mtmxt 1/1 Running 0 5s 10.0.0.42 node-exporter42 none none
[rootnode-exporter41 ~]# 6.验证CNI网络插件是否正常工作
[rootnode-exporter41 ~]# cat network-cni-test.yaml EOF
apiVersion: v1
kind: Pod
metadata:name: xiuxian-v1
spec:nodeName: node-exporter42containers:- image: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v1 name: xiuxian---apiVersion: v1
kind: Pod
metadata:name: xiuxian-v2
spec:nodeName: node-exporter43containers:- image: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v2name: xiuxian
EOF[rootnode-exporter41 ~]# kubectl apply -f oldboyedu-network-cni-test.yaml
pod/xiuxian-v1 created
pod/xiuxian-v2 created
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
xiuxian-v1 0/1 ContainerCreating 0 4s none node-exporter42 none none
xiuxian-v2 0/1 ContainerCreating 0 4s none node-exporter43 none none
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
xiuxian-v1 1/1 Running 0 9s 10.100.3.2 node-exporter42 none none
xiuxian-v2 1/1 Running 0 9s 10.100.0.2 node-exporter43 none none
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# curl 10.100.3.2
!DOCTYPE html
htmlheadmeta charsetutf-8/titleyinzhengjie apps v1/titlestylediv img {width: 900px;height: 600px;margin: 0;}/style/headbodyh1 stylecolor: green凡人修仙传 v1 /h1divimg src1.jpgdiv/body/html
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# curl 10.100.0.2
!DOCTYPE html
htmlheadmeta charsetutf-8/titleyinzhengjie apps v2/titlestylediv img {width: 900px;height: 600px;margin: 0;}/style/headbodyh1 stylecolor: red凡人修仙传 v2 /h1divimg src2.jpgdiv/body/html
[rootnode-exporter41 ~]# 7.开启自动补全功能
kubectl completion bash ~/.kube/completion.bash.inc
echo source $HOME/.kube/completion.bash.inc $HOME/.bashrc
source $HOME/.bashrc 18.CoreDNS部署及故障排查
1 下载资源清单
参考链接:https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/dns/coredns[rootnode-exporter41 ~]# wget http://192.168.16.253/Resources/Kubernetes/Add-ons/CoreDNS/coredns.yaml.base2 修改资源清单模板的关键字段
[rootnode-exporter41 ~]# sed -i /__DNS__DOMAIN__/s#__DNS__DOMAIN__#oldboyedu.com# coredns.yaml.base
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# sed -i /__DNS__MEMORY__LIMIT__/s#__DNS__MEMORY__LIMIT__#200Mi# coredns.yaml.base
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# sed -i /__DNS__SERVER__/s#__DNS__SERVER__#10.200.0.254# coredns.yaml.base
[rootnode-exporter41 ~]# 相关字段说明:__DNS__DOMAIN__DNS自定义域名要和你实际的K8S域名对应上。__DNS__MEMORY__LIMIT__CoreDNS组件的内存限制。__DNS__SERVER__DNS服务器的svc的CLusterIP地址。3 kubelet指定Pod的DNS服务器3.1 准备解析的配置文件
[rootnode-exporter41 ~]# cat /etc/kubernetes/resolv.conf EOF
nameserver 223.5.5.5
options edns0 trust-ad
search .
EOF[rootnode-exporter41 ~]# data_rsync.sh /etc/kubernetes/resolv.conf3.2.kubelet指定dns文件
[rootnode-exporter41 ~]# grep resolvConf /etc/kubernetes/kubelet-conf.yml
resolvConf: /etc/resolv.conf
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# sed -i /resolvConf/s#resolv.conf#kubernetes/resolv.conf# /etc/kubernetes/kubelet-conf.yml
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# grep resolvConf /etc/kubernetes/kubelet-conf.yml
resolvConf: /etc/kubernetes/resolv.conf
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# data_rsync.sh /etc/kubernetes/kubelet-conf.yml
[rootnode-exporter41 ~]# 3.3.K8S集群所有节点重启服务使得配置生效
systemctl restart kubelet.service 4.部署CoreDNS组件
[rootnode-exporter41 ~]# kubectl apply -f coredns.yaml.base
serviceaccount/coredns created
clusterrole.rbac.authorization.k8s.io/system:coredns created
clusterrolebinding.rbac.authorization.k8s.io/system:coredns created
configmap/coredns created
deployment.apps/coredns created
service/kube-dns created
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-5578c9dc84-wws5t 1/1 Running 0 10s 10.100.3.4 node-exporter42 none none
[rootnode-exporter41 ~]# 温馨提示:如果镜像下载失败可以手动导入。操作如下:wget http://192.168.16.253/Resources/Kubernetes/Add-ons/CoreDNS/oldboyedu-coredns-v1.12.0.tar.gzctr -n k8s.io i import oldboyedu-coredns-v1.12.0.tar.gz 5.验证DNS服务
[rootnode-exporter41 ~]# kubectl get svc -A
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.200.0.1 none 443/TCP 3h56m
kube-system kube-dns ClusterIP 10.200.0.254 none 53/UDP,53/TCP,9153/TCP 15m
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# dig 10.200.0.254 kube-dns.kube-system.svc.oldboyedu.com short
10.200.0.254
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# dig 10.200.0.254 kubernetes.default.svc.oldboyedu.com short
10.200.0.1
[rootnode-exporter41 ~]# 6.部署Pod验证默认的DNS服务器
[rootnode-exporter41 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
xiuxian-v1 1/1 Running 0 19s 10.100.3.5 node-exporter42 none none
xiuxian-v2 1/1 Running 0 19s 10.100.0.4 node-exporter43 none none
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# kubectl exec -it xiuxian-v1 -- cat /etc/resolv.conf
search default.svc.oldboyedu.com svc.oldboyedu.com oldboyedu.com
nameserver 10.200.0.254
options ndots:5
[rootnode-exporter41 ~]# 7.清除Pod环境
[rootnode-exporter41 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
xiuxian-v1 1/1 Running 0 2m20s 10.100.3.5 node-exporter42 none none
xiuxian-v2 1/1 Running 0 2m20s 10.100.0.4 node-exporter43 none none
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# kubectl delete pods --all
pod xiuxian-v1 deleted
pod xiuxian-v2 deleted
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# kubectl get pods -o wide
No resources found in default namespace.
[rootnode-exporter41 ~]#
19.验证 K8S 集群高可用
kubectl get no -o wide
查看集群状态
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
node-exporter41 Ready none 8h v1.32.4 10.0.0.41 none Ubuntu 22.04.4 LTS 5.15.0-135-generic containerd://1.6.36
node-exporter42 Ready none 8h v1.32.4 10.0.0.42 none Ubuntu 22.04.4 LTS 5.15.0-134-generic containerd://1.6.36
node-exporter43 Ready none 8h v1.32.4 10.0.0.43 none Ubuntu 22.04.4 LTS 5.15.0-135-generic containerd://1.6.36 五、附加组件部署
1.Helm
1.安装helm
[rootnode-exporter41 ~]# wget https://get.helm.sh/helm-v3.17.3-linux-amd64.tar.gz
[rootnode-exporter41 ~]# tar xf helm-v3.17.3-linux-amd64.tar.gz -C /usr/local/bin/ linux-amd64/helm --strip-components1
[rootnode-exporter41 ~]# helm version
version.BuildInfo{Version:v3.17.3, GitCommit:e4da49785aa6e6ee2b86efd5dd9e43400318262b, GitTreeState:clean, GoVersion:go1.23.7}# 同步到其他节点上
[rootnode-exporter41 ~]# data_rsync.sh /usr/local/bin/helm rsyncing node-exporter42: helm
命令执行成功!rsyncing node-exporter43: helm
命令执行成功!
[rootnode-exporter41 ~]# 2.所有节点配置helm的自动补全功能
helm completion bash /etc/bash_completion.d/helm
source /etc/bash_completion.d/helm
echo source /etc/bash_completion.d/helm ~/.bashrc
2.Ingress-nginx
github地址:https://github.com/kubernetes/ingress-nginx1.添加第三方仓库
[rootnode-exporter41 ~]# helm repo add ingress https://kubernetes.github.io/ingress-nginx
oldboyedu-ingress has been added to your repositories
[rootnode-exporter41 ~]# helm repo list
NAME URL
ingress https://kubernetes.github.io/ingress-nginx2.下载指定的Chart
[rootnode-exporter41 ~]# helm search repo ingress-nginx
NAME CHART VERSION APP VERSION DESCRIPTION
oldboyedu-ingress/ingress-nginx 4.12.1 1.12.1
Ingress controller for Kubernetes using NGINX a...
[rootnode-exporter41 ~]# helm pull oldboyedu-ingress/ingress-nginx --version 4.12.1 5.解压软件包并修改配置参数
[rootnode-exporter41 ~]# tar xf ingress-nginx-4.12.1.tgz [rootnode-exporter41 ~]# sed -ri /digest:/s^# ingress-nginx/values.yaml
[rootnode-exporter41 ~]# sed -i /hostNetwork:/s#false#true# ingress-nginx/values.yaml
[rootnode-exporter41 ~]# sed -i /dnsPolicy/s#ClusterFirst#ClusterFirstWithHostNet# ingress-nginx/values.yaml
[rootnode-exporter41 ~]# sed -i /kind/s#Deployment#DaemonSet# ingress-nginx/values.yaml
[rootnode-exporter41 ~]# sed -i /default:/s#false#true# ingress-nginx/values.yaml温馨提示:注意记得编辑values.yaml文件将admissionWebhooks功能禁用。即 enabled: false [rootnode-exporter41 ~]# grep -A 10 admissionWebhooks ingress-nginx/values.yaml admissionWebhooks:
..........enabled: false6.安装ingress-nginx
[rootnode-exporter41 ~]# helm upgrade --install myingress ingress-nginx -n ingress-nginx --create-namespace
Release myingress does not exist. Installing it now.7.验证Ingress-nginx是否安装成功
[rootnode-exporter41 ~]# helm -n ingress-nginx list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
myingress ingress-nginx 1 2025-04-23 21:16:21.595928036 0800 CST deployed ingress-nginx-4.12.1 1.12.1 [rootnode-exporter41 ~]# kubectl get ingressclass,deploy,svc,po -n ingress-nginx -o wide
NAME CONTROLLER PARAMETERS AGE
ingressclass.networking.k8s.io/nginx k8s.io/ingress-nginx none 2m7sNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/myingress-ingress-nginx-controller LoadBalancer 10.200.105.97 pending 80:44150/TCP,443:15236/TCP 2m7s app.kubernetes.io/componentcontroller,app.kubernetes.io/instancemyingress,app.kubernetes.io/nameingress-nginxNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/myingress-ingress-nginx-controller-7mnbk 1/1 Running 0 2m7s 10.0.0.42 node-exporter42 none none
pod/myingress-ingress-nginx-controller-hbpnp 1/1 Running 0 2m7s 10.0.0.41 node-exporter41 none none
pod/myingress-ingress-nginx-controller-jkndx 1/1 Running 0 2m7s 10.0.0.43 node-exporter43 none none8.访问测试因为我们的配置直接使用宿主机网络即可。
http://10.0.0.41/
http://10.0.0.42/
http://10.0.0.43/
3.metallb
1.下载metallb部署
[rootnode-exporter41 ~]# wget https://raw.githubusercontent.com/metallb/metallb/v0.14.9/config/manifests/metallb-native.yaml[rootnode-exporter41 ~]# kubectl apply -f metallb-native.yaml
namespace/metallb-system created
....2. 查看metallb的状态
[rootnode-exporter41 ~]# kubectl get all -o wide -n metallb-system
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/controller-bb5f47665-tjz84 1/1 Running 0 30s 10.100.3.3 node-exporter43 none none
pod/speaker-h6n9d 1/1 Running 0 30s 10.0.0.42 node-exporter42 none none
pod/speaker-m4kk7 1/1 Running 0 30s 10.0.0.41 node-exporter41 none none
pod/speaker-wxlwj 1/1 Running 0 30s 10.0.0.43 node-exporter43 none noneNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/metallb-webhook-service ClusterIP 10.200.119.247 none 443/TCP 30s componentcontrollerNAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE CONTAINERS IMAGES SELECTOR
daemonset.apps/speaker 3 3 3 3 3 kubernetes.io/oslinux 30s speaker quay.io/metallb/speaker:v0.14.9 appmetallb,componentspeakerNAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/controller 1/1 1 1 30s controller quay.io/metallb/controller:v0.14.9 appmetallb,componentcontrollerNAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.apps/controller-bb5f47665 1 1 1 30s controller quay.io/metallb/controller:v0.14.9 appmetallb,componentcontroller,pod-template-hashbb5f476653.创建MetalLB地址池
[rootnode-exporter41 ~]# cat metallb-ip-pool.yaml EOF
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:name: yinzhengjie-k8s-metallb-custom-poolnamespace: metallb-system
spec:addresses:- 10.0.0.150-10.0.0.180---apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:name: oldboyedunamespace: metallb-system
spec:ipAddressPools:- yinzhengjie-k8s-metallb-custom-pool
EOF
[rootnode-exporter41 ~]# kubectl apply -f metallb-ip-pool.yaml
ipaddresspool.metallb.io/yinzhengjie-k8s-metallb-custom-pool created
l2advertisement.metallb.io/oldboyedu created### 验证我们的ingress使用的LB是否出现了地址
[rootnode-exporter41 ~]# kubectl get svc -A
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.200.0.1 none 443/TCP 9h
ingress-nginx myingress-ingress-nginx-controller LoadBalancer 10.200.105.97 10.0.0.150 80:44150/TCP,443:15236/TCP 5m49s
4.metrics-server
部署文档https://github.com/kubernetes-sigs/metrics-server1 下载资源清单
[rootnode-exporter41 ~]# wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/high-availability-1.21.yaml2.编辑配置文件
[rootnode-exporter41 ~]# vim high-availability-1.21.yaml
...
114 apiVersion: apps/v1
115 kind: Deployment
116 metadata:
...
144 - args:
145 - --kubelet-insecure-tls # 不要验证Kubelets提供的服务证书的CA。不配置则会报错x509。
...### 验证所有节点是否都配置了--requestheader-allowed-namesfront-proxy-client参数
vim /lib/systemd/system/kube-apiserver.service
[Unit]
DescriptionJason Yins Kubernetes API Server
Documentationhttps://github.com/kubernetes/kubernetes
Afternetwork.target[Service]
ExecStart/usr/local/bin/kube-apiserver \--requestheader-allowed-namesfront-proxy-client \--v2 \2.如果没有配置参数添加后重启服务即可。
systemctl daemon-reload systemctl restart kube-apiserverss -ntl |grep 6443
LISTEN 0 16384 *:6443 *:* ss -ntl |grep 8443
LISTEN 0 2000 127.0.0.1:8443 0.0.0.0:*
LISTEN 0 2000 0.0.0.0:8443 0.0.0.0:* 3.部署metrics-server
[rootnode-exporter41 ~]# kubectl apply -f high-availability-1.21.yaml4.验证测试
[rootnode-exporter41 ~]# kubectl get pods -o wide -n kube-system -l k8s-appmetrics-server
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
metrics-server-9c85f7647-8lfrz 1/1 Running 0 28s 10.100.1.3 node-exporter42 none none
metrics-server-9c85f7647-kpzjz 1/1 Running 0 28s 10.100.3.4 node-exporter43 none none
[rootnode-exporter41 ~]# kubectl top no
NAME CPU(cores) CPU(%) MEMORY(bytes) MEMORY(%)
node-exporter41 109m 5% 1064Mi 18%
node-exporter42 116m 5% 866Mi 14%
node-exporter43 218m 10% 1169Mi 20%
5.基于helm部署Dashboard
参考链接:https://github.com/kubernetes/dashboard1.添加Dashboard的仓库地址
[rootnode-exporter41 ~]# helm repo add kubernetes-dashboard https://kubernetes.github.io/dashboard/
kubernetes-dashboard has been added to your repositories
[rootnode-exporter41 ~]# helm repo list
NAME URL
oldboyedu-ingress https://kubernetes.github.io/ingress-nginx
kubernetes-dashboard https://kubernetes.github.io/dashboard/ 2.安装Dashboard
[rootnode-exporter41 ~]# helm upgrade --install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard --create-namespace --namespace kubernetes-dashboard[rootnode-exporter41 ~]# tar xf kubernetes-dashboard-7.12.0.tgz
[rootnode-exporter41 ~]#
[rootnode-exporter41 ~]# ll kubernetes-dashboard
total 56
drwxr-xr-x 4 root root 4096 Apr 23 21:46 ./
drwx------ 10 root root 4096 Apr 23 21:46 ../
-rw-r--r-- 1 root root 497 Apr 16 23:19 Chart.lock
drwxr-xr-x 6 root root 4096 Apr 23 21:46 charts/
-rw-r--r-- 1 root root 982 Apr 16 23:19 Chart.yaml
-rw-r--r-- 1 root root 948 Apr 16 23:19 .helmignore
-rw-r--r-- 1 root root 8209 Apr 16 23:19 README.md
drwxr-xr-x 10 root root 4096 Apr 23 21:46 templates/
-rw-r--r-- 1 root root 13729 Apr 16 23:19 values.yaml[rootnode-exporter41 ~]# helm upgrade --install mywebui kubernetes-dashboard --create-namespace --namespace kubernetes-dashboard
Release mywebui does not exist. Installing it now.
NAME: mywebui
LAST DEPLOYED: Wed Apr 23 21:54:09 2025
NAMESPACE: kubernetes-dashboard
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
*************************************************************************************************
*** PLEASE BE PATIENT: Kubernetes Dashboard may need a few minutes to get up and become ready ***
*************************************************************************************************Congratulations! You have just installed Kubernetes Dashboard in your cluster.To access Dashboard run:kubectl -n kubernetes-dashboard port-forward svc/mywebui-kong-proxy 8443:443NOTE: In case port-forward command does not work, make sure that kong service name is correct.Check the services in Kubernetes Dashboard namespace using:kubectl -n kubernetes-dashboard get svcDashboard will be available at:https://localhost:84433.查看部署信息
[rootnode-exporter41 ~]# helm -n kubernetes-dashboard list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
mywebui kubernetes-dashboard 1 2025-04-23 21:54:09.158789016 0800 CST deployed kubernetes-dashboard-7.12.0 [rootnode-exporter41 ~]# kubectl -n kubernetes-dashboard get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
mywebui-kong-6dd8f649b4-2gxqw 1/1 Running 0 72s 10.100.1.4 node-exporter42 none none
mywebui-kubernetes-dashboard-api-5445d676c7-ggrm8 1/1 Running 0 72s 10.100.1.5 node-exporter42 none none
mywebui-kubernetes-dashboard-auth-8687596fbd-xs8w9 1/1 Running 0 72s 10.100.0.4 node-exporter41 none none
mywebui-kubernetes-dashboard-metrics-scraper-6ccc47c6ff-cwnq4 1/1 Running 0 72s 10.100.3.5 node-exporter43 none none
mywebui-kubernetes-dashboard-web-584df9444c-n9sgj 1/1 Running 0 72s 10.100.0.5 node-exporter41 none none
[rootnode-exporter41 ~]# 4.修改svc的类型
[rootnode-exporter41 ~]# kubectl edit svc -n kubernetes-dashboard mywebui-kong-proxy type: NodePort
service/mywebui-kong-proxy edited[rootnode-exporter41 ~]# kubectl -n kubernetes-dashboard get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
mywebui-kong-6dd8f649b4-2gxqw 1/1 Running 0 3m18s 10.100.1.4 node-exporter42 none none
mywebui-kubernetes-dashboard-api-5445d676c7-ggrm8 1/1 Running 0 3m18s 10.100.1.5 node-exporter42 none none
mywebui-kubernetes-dashboard-auth-8687596fbd-xs8w9 1/1 Running 0 3m18s 10.100.0.4 node-exporter41 none none
mywebui-kubernetes-dashboard-metrics-scraper-6ccc47c6ff-cwnq4 1/1 Running 0 3m18s 10.100.3.5 node-exporter43 none none
mywebui-kubernetes-dashboard-web-584df9444c-n9sgj 1/1 Running 0 3m18s 10.100.0.5 node-exporter41 none none[rootnode-exporter41 ~]# kubectl get svc -n kubernetes-dashboard mywebui-kong-proxy
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mywebui-kong-proxy NodePort 10.200.163.82 none 443:7118/TCP 3m27s5.访问WebUI
https://10.0.0.43:7118/#/login 6.创建登录账号6.1 创建sa
[rootnode-exporter41 ~]# kubectl create serviceaccount admin
serviceaccount/admin created
[rootnode-exporter41 ~]# 6.2 将sa和CLuster-admin进行绑定
[rootnode-exporter41 ~]# kubectl create clusterrolebinding dashboard-admin --serviceaccountdefault:admin --clusterrolecluster-admin
clusterrolebinding.rbac.authorization.k8s.io/dashboard-admin created
[rootnode-exporter41 ~]# [rootnode-exporter41 ~]# kubectl create token admin
eyJhbGciOiJSUzI1NiIsImtpZCI6IlJEV1R3UmY2Wm9JSmJrdF96amVBdFZQeUZyM2p5Z1BPY1VnVnJjWE9tR2cifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLm9sZGJveWVkdS5jb20iXSwiZXhwIjoxNzQ1NDIwNDE4LCJpYXQiOjE3NDU0MTY4MTgsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5vbGRib3llZHUuY29tIiwianRpIjoiYjk5MzViMmQtNzI5YS00YmIwLThlNTEtYzI2OWUzMmMxNThlIiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJkZWZhdWx0Iiwic2VydmljZWFjY291bnQiOnsibmFtZSI6ImFkbWluIiwidWlkIjoiNDFkN2I5OWEtYzFlMC00ZGEwLWExZjMtZDg0OWU1NDFiZjlhIn19LCJuYmYiOjE3NDU0MTY4MTgsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpkZWZhdWx0OmFkbWluIn0.kcYS6zxk7bEoqwzQvIWasJgMNJRRZyiECb00VV1ty1oTsj6fk5jNCc2tQkSaHoEFp6WvTcP9-Qc99C00RNCLFFnmLiTrRkW9zGP8YfmevJUCdm3wbJ1qWyimdEcCCQcllZTMUMtvYr9gPk2kS6kCijokFv4sXWL8VsMUYg32gEIaz_o6KOGfR9BwfhQyzkQraIrp5M2-268kHwDTMdp-73C85IK7Uc4OuP93qZFCW961RHkEZY6IeXHbKwZx215J9PNxHdWxo8tWZhqV8YUX4ggeXEJUhDUlrApEJMJyC_IVQq9mn3dKv3p6Odwo6BNWKLVFR1GeswTZXic5OVUzZw 六、总结
通过以上步骤成功部署了一个基于 Ubuntu 22.04 的 K8S 1.32 高可用集群。整个过程涵盖了从基础环境准备到 K8S 集群的完整部署包括 Etcd 高可用集群的搭建、K8S 组件的部署、证书的生成与配置、高可用组件的安装与验证等关键环节。
这一部署过程详细展示了如何构建一个多 master 的高可用 K8S 集群确保了在单节点故障的情况下集群仍能正常运行提高了系统的可靠性和稳定性。可用于企业级应用的部署为业务的连续性和高可用性提供了有力保障。
通过精心设计的网络架构和组件配置K8S 集群能够在节点故障时自动进行负载转移保持服务的不间断运行。同时采用的证书管理方案确保了集群内部通信的安全性为后续的应用部署和扩展奠定了坚实的基础。
这一实践不仅验证了 K8S 高可用架构的可行性还积累了宝贵的部署经验为后续复杂应用在 K8S 上的部署和管理提供了有力支持。未来可以在这一基础上进一步扩展如部署 Ingress 控制器、网络存储插件等以满足更多样化的企业应用需求。
