技术开发包括哪些内容,重庆seo扣费,大数据查询平台,工商登记查询网官网温故知新 #x1f4da;第三章 Kubernetes各组件部署#x1f4d7;安装kubectl#xff08;可直接跳转到安装kubeadm章节#xff0c;直接全部安装了#xff09;#x1f4d5;下载kubectl安装包#x1f4d5;执行kubectl安装#x1f4d5;验证kubectl #x1f4d7;安装kubead… 温故知新 第三章 Kubernetes各组件部署安装kubectl可直接跳转到安装kubeadm章节直接全部安装了下载kubectl安装包执行kubectl安装验证kubectl 安装kubeadm添加yum存储库配置文件kubernetes.repo安装kubeadm kubelet kubectl 前面安装kubectl 可以放到这里一块安装查看版本信息启动kubeletkubelet启动、查看状态、日志命令 其他机器Kubernetes组件安装关闭防火墙、swap、selinux创建sudo安装用户kubernetes添加kubernetes对应的yum存储库配置文件安装kubeadm kubelet kubectl安装命令验证 第四章 kubernetes集群部署安装dockerr1、r2、r3都要安装yum 方式安装 安装cri-dockerdr1、r2、r3都要安装下载rpm安装包通过rpm安装启动cri-dockerd 初始化K8S集群配置/etc/hostsr1、r2、r3三台机器配置初始化系统参数module参数sysctl 参数 Master节点初始化对应r1机器kubeadm initadmin.conf 安装网络插件r1、r2、r3机器Node节点假如到Master对应r2、r3机器 总结⁉️问题记录❓问题一yum安装软件提示“网络不可达”❗解决方式一重启虚拟机网络sudo systemctl restart network❕其他替换镜像地址为阿里云的命令 ❓问题二容器运行时名词解释❓问题三获取阿里云镜像加速器地址步骤❓问题四kubeadm init执行失败❓错误详情一detected that the sandbox image registry.k8s.io/pause:3.6 of the container runtime is inconsistent with that used by kubeadm. It is recommended that using registry.aliyuncs.com/google_containers/pause:3.9 as the CRI sandbox image❓错误详情二[kubelet-check] Initial timeout of 40s passed.❗最终解决修改vi /usr/lib/systemd/system/cri-docker.service ❓问题五kubeadm join执行失败❓错误详情一[WARNING Hostname]: hostname renxiaozhao03could not be reached❗解决方式添加--node-name参数指定主机名❓错误详情二超时报错error execution phase kubelet-start: error uploading crisocket: nodes k8s-node01 not found❗解决方式使用kubeadm reset重置之后再添加join弯路历程 上一集K8S最新版本集群部署(v1.28) 容器引擎Docker部署上 *️⃣主目录温故知新专栏 下一集Kubernetes可视化管理工具Kuboard部署使用及k8s常用命令梳理记录 第三章 Kubernetes各组件部署
安装kubectl可直接跳转到安装kubeadm章节直接全部安装了
kubectl 是使用 Kubernetes API 与 Kubernetes 集群的控制面进行通信的命令行工具。详见官网安装步骤
下载kubectl安装包
kubectl v1.28.1下载
执行kubectl安装
[kubernetesrenxiaozhao01 ~]$ sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
[kubernetesrenxiaozhao01 ~]$
[kubernetesrenxiaozhao01 ~]$ kubectl version --client
Client Version: v1.28.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
[kubernetesrenxiaozhao01 ~]$
[kubernetesrenxiaozhao01 ~]$
就这么简单之前傻傻的通过curl下载估计哪里卡住了一个半小时都没下载好网络环境好的话当然还是直接curl -LO https://dl.k8s.io/release/v1.28.1/bin/linux/amd64/kubectl方便
验证kubectl
执行kubectl version --client或者kubectl version --client --outputyaml
[kubernetesrenxiaozhao01 ~]$ kubectl version --client --outputyaml
clientVersion:buildDate: 2023-08-24T11:23:10Zcompiler: gcgitCommit: 8dc49c4b984b897d423aab4971090e1879eb4f23gitTreeState: cleangitVersion: v1.28.1goVersion: go1.20.7major: 1minor: 28platform: linux/amd64
kustomizeVersion: v5.0.4-0.20230601165947-6ce0bf390ce3[kubernetesrenxiaozhao01 ~]$
[kubernetesrenxiaozhao01 ~]$ kubectl version --client
Client Version: v1.28.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
[kubernetesrenxiaozhao01 ~]$
[kubernetesrenxiaozhao01 ~]$ 安装kubeadm
官网详情
添加yum存储库配置文件kubernetes.repo
cat EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
nameKubernetes
baseurlhttp://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled1
gpgcheck0
repo_gpgcheck1
gpgkeyhttp://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpghttp://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
excludekubelet kubeadm kubectl
EOF安装kubeadm kubelet kubectl 前面安装kubectl 可以放到这里一块安装
安装命令sudo yum install -y kubelet kubeadm kubectl --disableexcludeskubernetes
这里不会重复安装对于之前已经安装过的kubectlyum安装时会自动跳过
查看版本信息
[kubernetesrenxiaozhao01 ~]$ kubelet --version
Kubernetes v1.28.1
[kubernetesrenxiaozhao01 ~]$ kubeadm version
kubeadm version: version.Info{Major:1, Minor:28, GitVersion:v1.28.1, GitCommit:8dc49c4b984b897d423aab4971090e1879eb4f23, GitTreeState:clean, BuildDate:2023-08-24T11:21:51Z, GoVersion:go1.20.7, Compiler:gc, Platform:linux/amd64}
[kubernetesrenxiaozhao01 ~]$
[kubernetesrenxiaozhao01 ~]$ 启动kubelet
执行sudo systemctl enable --now kubelet命令
kubelet运行在每个节点上并负责管理和监控节点上的容器。它与Kubernetes Master节点通信接收来自Master节点的指令并根据指令来创建、销毁和管理容器。enable将kubelet服务设置为开机启动。而通过添加--now选项kubelet服务将立即启动
kubelet启动、查看状态、日志命令 查看状态systemctl status kubelet kubelet 现在每隔几秒就会重启因为它陷入了一个等待 kubeadm 指令的死循环。 在使用kubeadm初始化Kubernetes集群时需要在Master节点上运行一些kubeadm命令来完成初始化过程。其中一个命令是kubeadm init它将生成一个用于加入其他节点的令牌。kubelet服务在启动时会检查是否存在这个令牌如果没有找到它会陷入一个等待kubeadm指令的死循环。 手动启动systemctl start kubelet 查看日志sudo journalctl -u kubelet
其他机器Kubernetes组件安装
到这里已经在r1机器安装完了Kubernetes核心组件参照上集主要就是关闭防火墙那些操作和本集安装kubeadm操作覆盖了kubeadm、kubelet、kubectl三个核心组件此处做个精简版概括继续在r2、r3机器安装机器信息在上集第二章有介绍。
关闭防火墙、swap、selinux
关闭防火墙systemctl stop firewalld.service
systemctl status firewalld.service
systemctl disable firewalld.service关闭swap 修改vi /etc/fstab注释掉/dev/mapper/centos-swap swap...这一行 关闭selinux 修改vi /etc/sysconfig/selinuxSELINUXenforcing改为SELINUXdisabled 重启虚拟机生效 执行reboot或者shutdown -r now
创建sudo安装用户kubernetes
useradd kubernetes -d /home/kubernetes
passwd kubernetes
echo kubernetes ALL(ALL) NOPASSWD: ALL /etc/sudoers添加kubernetes对应的yum存储库配置文件
cat EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
nameKubernetes
baseurlhttp://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled1
gpgcheck0
repo_gpgcheck1
gpgkeyhttp://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpghttp://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
excludekubelet kubeadm kubectl
EOF安装kubeadm kubelet kubectl
sudo yum install -y kubelet kubeadm kubectl --disableexcludeskubernetes安装命令验证
kubelet --version
kubeadm version
kubectl version --client第四章 kubernetes集群部署
官网创建集群说明
kubeadm init和kubeadm join命令在默认情况下会自动下载所需的Docker镜像。这些镜像包括Kubernetes组件如kube-apiserver、kube-controller-manager、kube-scheduler等以及其他一些常用的容器镜像如CoreDNS等。使用kubeadm init和kubeadm join命令来初始化和加入集群并且不需要自定义镜像源kubeadm init和kubeadm join命令会自动处理Docker的安装和配置。
尝试了一下直接执行kubeadm init反正都是错所以还是在r1、r2、r3上面手动安装docker相关组件吧 安装dockerr1、r2、r3都要安装
Kubernetes是一个容器编排平台用于管理和编排容器化应用程序。它可以与多种容器运行时集成包括Docker、containerd、CRI-O等。安装Kubernetes时默认情况下会使用Docker作为容器运行时这是因为Docker是目前最广泛使用的容器运行时之一具有广泛的社区支持和成熟的生态系统。因此大多数Kubernetes安装指南和教程都会建议先安装Docker。
总结起来Kubernetes和Docker是相互独立、不是必须绑定在一起的。但是正常在安装Kubernetes时默认都是先安装Docker容器
yum 方式安装
yum 这种方式的安装适用于大多数情况它会自动获取最新的稳定版本
下面的yum-config-manager是yum-utils中的一个工具sudo yum install -y yum-utils设置docker社区版存储库sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo安装dockersudo yum install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin启动dockersudo systemctl start docker设置docker开启启动虚拟机会经常重启为了减少麻烦最好设置开机启动sudo systemctl enable docker查看docker状态sudo systemctl status docker查看docker版本docker version[kubernetesrenxiaozhao01 ~]$ docker version
Client: Docker Engine - CommunityVersion: 24.0.5API version: 1.43Go version: go1.20.6Git commit: ced0996Built: Fri Jul 21 20:39:02 2023OS/Arch: linux/amd64Context: default
permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/version: dial unix /var/run/docker.sock: connect: permission denied
[kubernetesrenxiaozhao01 ~]$ 配置镜像加速器参照文章下面的问题三支付宝登录一下阿里云镜像地址就获取到了 sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json -EOF
{registry-mirrors: [https://xxxx.mirror.aliyuncs.com]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker执行sudo docker info可以看到刚刚配置的加速器信息
安装cri-dockerdr1、r2、r3都要安装
官网地址 cri-dockerd 是一种基于 Docker 的容器运行时实现它通过实现 CRI 接口使 Kubernetes 能够与 Docker 进行交互创建和管理容器
下载rpm安装包
官网说明 Git地址直接点击下载丢到服务器网络好的话可以直接服务器执行sudo wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.4/cri-dockerd-0.3.4-3.el7.x86_64.rpm进行下载下载挺慢… 这次反而在环境上下载快一点 心累…明晚继续 cri-dockerd-0.3.4-3.el7.x86-64.rpm快速下载地址
通过rpm安装
没有rpm命令可以执行yum - y intall rpm 安装该命令
通过rpm安装sudo rpm -ivh cri-dockerd-0.3.4-3.el7.x86_64.rpm-i 表示安装软件包-v 表示显示详细的安装过程-h 表示显示进度条。
启动cri-dockerd
# 重载系统守护进程
sudo systemctl daemon-reload
# 设置cri-dockerd自启动
sudo systemctl enable cri-docker.socket cri-docker
# 启动cri-dockerd
sudo systemctl start cri-docker.socket cri-docker
# 查看cri-dockerd状态
sudo systemctl status cri-docker.socket初始化K8S集群
配置/etc/hostsr1、r2、r3三台机器
名称添加k8s标识方便区分日后部署hadoop集群可以叫hp-master、hp-slave01、hp-slave02
[kubernetesrenxiaozhao01 ~]$ sudo vi /etc/hosts
[kubernetesrenxiaozhao01 ~]$ sudo cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.17.17 k8s-master
192.168.17.18 k8s-node01
192.168.17.19 k8s-node02
[kubernetesrenxiaozhao01 ~]$
配置初始化系统参数
module参数 内核模块配置文件 cat EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF加载内核模块 overlay 模块是用于支持容器的联合文件系统的 br_netfilter 模块是用于支持 Kubernetes 网络的桥接和网络过滤功能的。 加载命令 sudo modprobe overlay
sudo modprobe br_netfilter验证加载是否成功 lsmod | grep br_netfilter
lsmod | grep overlay sysctl 参数
net.bridge.bridge-nf-call-iptables 用于启用桥接网络对 iptables 的调用net.bridge.bridge-nf-call-ip6tables 用于启用桥接网络对 ip6tables 的调用net.ipv4.ip_forward 用于启用 IP 转发参数 配置文件 cat EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables 1
net.bridge.bridge-nf-call-ip6tables 1
net.ipv4.ip_forward 1
EOF生效配置 sudo sysctl --system查看配置 sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forwardMaster节点初始化对应r1机器
kubeadm init 官网地址 初始化命令 sudo kubeadm init --node-namek8s-master --image-repositoryregistry.aliyuncs.com/google_containers --cri-socketunix:///var/run/cri-dockerd.sock --apiserver-advertise-address192.168.17.17 --pod-network-cidr10.244.0.0/16 --service-cidr10.96.0.0/12kubeadm init 命令中的参数解释上面的官网地址中也有解释
--node-namek8s-master: 指定k8s集群中Master节点的名称和上面/etc/hosts配置中192.168.17.17 k8s-master名称一致。--image-repositoryregistry.aliyuncs.com/google_containers: 指定容器镜像的仓库地址。--cri-socketunix:///var/run/cri-dockerd.sock: 指定容器运行时的 CRIContainer Runtime Interface套接字路径Docker 的配置文件路径通常是 /etc/docker/daemon.jsonCRI 套接字路径通常不会在 Docker 的配置文件中直接指定。Kubernetes一般会使用默认的 Docker 套接字路径 /var/run/docker.sock。--apiserver-advertise-address192.168.17.17: 指定Kubernetes API Server的地址即Master节点的IP地址。和上面/etc/hosts配置中192.168.17.17 k8s-master的IP一致--pod-network-cidr10.244.0.0/16: 指定 Pod 网络的 CIDRClassless Inter-Domain Routing地址范围。默认使用这个即可。--service-cidr10.96.0.0/12: 指定服务网络的 CIDR 地址范围需要选择一个未被使用的 CIDR 地址范围一般默认使用这个即可。执行成功如下差点崩溃了卡了好久详见问题四 [kubernetesrenxiaozhao01 ~]$ sudo kubeadm init --node-namek8s-master --image-repositoryregistry.aliyuncs.com/google_containers --cri-socketunix:///var/run/cri-dockerd.sock --apiserver-advertise-address192.168.17.17 --pod-network-cidr10.244.0.0/16 --service-cidr10.96.0.0/12
[init] Using Kubernetes version: v1.28.1
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using kubeadm config images pull
[certs] Using certificateDir folder /etc/kubernetes/pki
[certs] Generating ca certificate and key
[certs] Generating apiserver certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.17.17]
[certs] Generating apiserver-kubelet-client certificate and key
[certs] Generating front-proxy-ca certificate and key
[certs] Generating front-proxy-client certificate and key
[certs] Generating etcd/ca certificate and key
[certs] Generating etcd/server certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.17.17 127.0.0.1 ::1]
[certs] Generating etcd/peer certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.17.17 127.0.0.1 ::1]
[certs] Generating etcd/healthcheck-client certificate and key
[certs] Generating apiserver-etcd-client certificate and key
[certs] Generating sa key and public key
[kubeconfig] Using kubeconfig folder /etc/kubernetes
[kubeconfig] Writing admin.conf kubeconfig file
[kubeconfig] Writing kubelet.conf kubeconfig file
[kubeconfig] Writing controller-manager.conf kubeconfig file
[kubeconfig] Writing scheduler.conf kubeconfig file
[etcd] Creating static Pod manifest for local etcd in /etc/kubernetes/manifests
[control-plane] Using manifest folder /etc/kubernetes/manifests
[control-plane] Creating static Pod manifest for kube-apiserver
[control-plane] Creating static Pod manifest for kube-controller-manager
[control-plane] Creating static Pod manifest for kube-scheduler
[kubelet-start] Writing kubelet environment file with flags to file /var/lib/kubelet/kubeadm-flags.env
[kubelet-start] Writing kubelet configuration to file /var/lib/kubelet/config.yaml
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory /etc/kubernetes/manifests. This can take up to 4m0s
[apiclient] All control plane components are healthy after 16.526647 seconds
[upload-config] Storing the configuration used in ConfigMap kubeadm-config in the kube-system Namespace
[kubelet] Creating a ConfigMap kubelet-config in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node k8s-master as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node k8s-master as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Using token: 4ydg4t.7cjjm52hd4p86gmk
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the cluster-info ConfigMap in the kube-public namespace
[kubelet-finalize] Updating /etc/kubernetes/kubelet.conf to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxyYour Kubernetes control-plane has initialized successfully!To start using your cluster, you need to run the following as a regular user:mkdir -p $HOME/.kubesudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/configAlternatively, if you are the root user, you can run:export KUBECONFIG/etc/kubernetes/admin.confYou should now deploy a pod network to the cluster.
Run kubectl apply -f [podnetwork].yaml with one of the options listed at:https://kubernetes.io/docs/concepts/cluster-administration/addons/Then you can join any number of worker nodes by running the following on each as root:kubeadm join 192.168.17.17:6443 --token 4ydg4t.7cjjm52hd4p86gmk \--discovery-token-ca-cert-hash sha256:ee2c3ae1c2d702b77a0b52f9dafe734aa7e25f33c44cf7fa469c1adc8c176be1
admin.conf 参照上面初始化信息还需要执行以下命令(kubernetes用户如果用root执行的命令):[kubernetesrenxiaozhao01 ~]$
[kubernetesrenxiaozhao01 ~]$ mkdir -p $HOME/.kube
[kubernetesrenxiaozhao01 ~]$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[kubernetesrenxiaozhao01 ~]$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
[kubernetesrenxiaozhao01 ~]$
[kubernetesrenxiaozhao01 ~]$ 如果用root执行的命令: 直接执行export KUBECONFIG/etc/kubernetes/admin.conf临时生效建议配置放到环境变量文件非root用户不需要再配置环境变量
安装网络插件r1、r2、r3机器
应该是必须安装一开始没安装执行下 kubeadm join也可以添加成功但是状态应该不对后来 kubeadm join 添加节点名称参数时候一直不成功然后尝试安装网络插件flannel下载地址简单记录下吧
下载kube-flannel.yml指定网络接口名称搜索kube-subnet-mgr 下面添加 - --ifaceens33对应自己的固定IP网关 直接执行kubectl apply -f kube-flannel.yml如果非root用户添加sudo会报错很多加sudo执行都会报错localhost:8080 was refused去掉就好了有待研究[kubernetesrenxiaozhao03 ~]$ kubectl apply -f kube-flannel.yml
namespace/kube-flannel unchanged
serviceaccount/flannel unchanged
clusterrole.rbac.authorization.k8s.io/flannel unchanged
clusterrolebinding.rbac.authorization.k8s.io/flannel unchanged
configmap/kube-flannel-cfg unchanged
daemonset.apps/kube-flannel-ds unchanged
[kubernetesrenxiaozhao03 ~]$
[kubernetesrenxiaozhao03 ~]$
[kubernetesrenxiaozhao03 ~]$
[kubernetesrenxiaozhao03 ~]$ sudo kubectl apply -f kube-flannel.yml
The connection to the server localhost:8080 was refused - did you specify the right host or port?
[kubernetesrenxiaozhao03 ~]$
[kubernetesrenxiaozhao03 ~]$
Node节点假如到Master对应r2、r3机器 同步master上的 .kube到node上面 scp -r .kube 192.168.17.18:/home/kubernetes/ 同步完验证下 执行上文生成的join指令 sudo kubeadm join 192.168.17.17:6443 --token 4ydg4t.7cjjm52hd4p86gmk --discovery-token-ca-cert-hash sha256:ee2c3ae1c2d702b77a0b52f9dafe734aa7e25f33c44cf7fa469c1adc8c176be1 --cri-socketunix:///var/run/cri-dockerd.sock添加成功后查看节点信息kubectl get nodes 有个节点没准备好呀有缘再说吧累了不过好像可以确认网络插件属于必装比如flannel 没忍住排查了一下反正现在好了就是按照下面的问题四修改了一下cri-docker.service然后反复重启了几次 [kubernetesrenxiaozhao03 ~]$ sudo systemctl stop cri-docker
[kubernetesrenxiaozhao03 ~]$ #怎么修改参照问题四
[kubernetesrenxiaozhao03 ~]$ sudo vi /usr/lib/systemd/system/cri-docker.service
[kubernetesrenxiaozhao03 ~]$ sudo systemctl daemon-reload
[kubernetesrenxiaozhao03 ~]$ sudo systemctl start cri-docker
[kubernetesrenxiaozhao03 ~]$ sudo systemctl restart kubelet docker cri-docker
[kubernetesrenxiaozhao03 ~]$
[kubernetesrenxiaozhao03 ~]$
[kubernetesrenxiaozhao03 ~]$ sudo systemctl restart kubelet docker cri-docker
[kubernetesrenxiaozhao03 ~]$ kubectl get pod -A|grep kube-flannel-ds-rc8vq
kube-flannel kube-flannel-ds-rc8vq 1/1 Running 2 (33s ago) 164m
[kubernetesrenxiaozhao03 ~]$ 总结
略
⁉️问题记录
❓问题一yum安装软件提示“网络不可达” [kubernetesrenxiaozhao01 ~]$ sudo yum -y install lrzsz
已加载插件fastestmirror, langpacks
Loading mirror speeds from cached hostfile
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release7archx86_64repoosinfrastock error was
14: curl#7 - Failed to connect to 2600:1f16:c1:5e01:4180:6610:5482:c1c0: 网络不可达One of the configured repositories failed (未知),and yum doesnt have enough cached data to continue. At this point the onlysafe thing yum can do is fail. There are a few ways to work fix this:1. Contact the upstream for the repository and get them to fix the problem.2. Reconfigure the baseurl/etc. for the repository, to point to a workingupstream. This is most often useful if you are using a newerdistribution release than is supported by the repository (and thepackages for the previous distribution release still work).3. Run the command with the repository temporarily disabledyum --disablereporepoid ...4. Disable the repository permanently, so yum wont use it by default. Yumwill then just ignore the repository until you permanently enable itagain or use --enablerepo for temporary usage:yum-config-manager --disable repoidorsubscription-manager repos --disablerepoid5. Configure the failing repository to be skipped, if it is unavailable.Note that yum will try to contact the repo. when it runs most commands,so will have to try and fail each time (and thus. yum will be be muchslower). If it is a very temporary problem though, this is often a nicecompromise:yum-config-manager --save --setoptrepoid.skip_if_unavailabletrueCannot find a valid baseurl for repo: base/7/x86_64
❗解决方式一重启虚拟机网络sudo systemctl restart network
从公司回到家切换了网络虚拟机已经重启了ping www.baidu.com也是通的后面又试了一下发现不通了突然想起来中间切换过一次无线网络重启网络就好了原因还是本地网络不好不稳长城宽带
[kubernetesrenxiaozhao01 yum.repos.d]$ curl https://www.baidu.com/
curl: (7) Failed connect to www.baidu.com:443; 没有到主机的路由执行重启命令sudo systemctl restart network
❕其他替换镜像地址为阿里云的命令
sudo curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo❓问题二容器运行时名词解释
容器运行时用于指代用于创建、打包和运行容器的软件组件 容器运行时是负责管理和执行容器的主要组件 负责加载和运行容器镜像创建和管理容器的生命周期并提供与容器交互的接口负责处理容器的资源隔离、网络和存储配置等方面 常见的容器运行时包括Docker、containerd、CRI-O等
❓问题三获取阿里云镜像加速器地址步骤
获取镜像加速器地址方法 直接支付宝扫码登录阿里云容器镜像服务
❓问题四kubeadm init执行失败
[init] Using Kubernetes version: v1.28.1
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using kubeadm config images pull
W0829 19:08:22.039382 55875 checks.go:835] detected that the sandbox image registry.k8s.io/pause:3.6 of the container runtime is inconsistent with that used by kubeadm. It is recommended that using registry.aliyuncs.com/google_containers/pause:3.9 as the CRI sandbox image.
[certs] Using certificateDir folder /etc/kubernetes/pki
[certs] Generating ca certificate and key
[certs] Generating apiserver certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.17.17]
[certs] Generating apiserver-kubelet-client certificate and key
[certs] Generating front-proxy-ca certificate and key
[certs] Generating front-proxy-client certificate and key
[certs] Generating etcd/ca certificate and key
[certs] Generating etcd/server certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.17.17 127.0.0.1 ::1]
[certs] Generating etcd/peer certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master localhost] and IPs [192.168.17.17 127.0.0.1 ::1]
[certs] Generating etcd/healthcheck-client certificate and key
[certs] Generating apiserver-etcd-client certificate and key
[certs] Generating sa key and public key
[kubeconfig] Using kubeconfig folder /etc/kubernetes
[kubeconfig] Writing admin.conf kubeconfig file
[kubeconfig] Writing kubelet.conf kubeconfig file
[kubeconfig] Writing controller-manager.conf kubeconfig file
[kubeconfig] Writing scheduler.conf kubeconfig file
[etcd] Creating static Pod manifest for local etcd in /etc/kubernetes/manifests
[control-plane] Using manifest folder /etc/kubernetes/manifests
[control-plane] Creating static Pod manifest for kube-apiserver
[control-plane] Creating static Pod manifest for kube-controller-manager
[control-plane] Creating static Pod manifest for kube-scheduler
[kubelet-start] Writing kubelet environment file with flags to file /var/lib/kubelet/kubeadm-flags.env
[kubelet-start] Writing kubelet configuration to file /var/lib/kubelet/config.yaml
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory /etc/kubernetes/manifests. This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.Unfortunately, an error has occurred:timed out waiting for the conditionThis error is likely caused by:- The kubelet is not running- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:- systemctl status kubelet- journalctl -xeu kubeletAdditionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:- crictl --runtime-endpoint unix:///var/run/cri-dockerd.sock ps -a | grep kube | grep -v pauseOnce you have found the failing container, you can inspect its logs with:- crictl --runtime-endpoint unix:///var/run/cri-dockerd.sock logs CONTAINERID
error execution phase wait-control-plane: couldnt initialize a Kubernetes cluster
To see the stack trace of this error execute with --v5 or higher❓错误详情一detected that the sandbox image registry.k8s.io/pause:3.6 of the container runtime is inconsistent with that used by kubeadm. It is recommended that using registry.aliyuncs.com/google_containers/pause:3.9 as the CRI sandbox image
❓错误详情二[kubelet-check] Initial timeout of 40s passed.
❗最终解决修改vi /usr/lib/systemd/system/cri-docker.service
错误详情一通过修改cri-docker.service解决错误详情二由错误详情一导致 解决步骤 停止cri-docker服务sudo systemctl stop cri-docker 编辑vi /usr/lib/systemd/system/cri-docker.service 找到ExecStart在后面添加--pod-infra-container-imageregistry.aliyuncs.com/google_containers/pause:3.9ExecStart/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --pod-infra-container-imageregistry.aliyuncs.com/google_containers/pause:3.9重新加载服务sudo systemctl daemon-reload 启动cri-docker服务sudo systemctl start cri-docker 查看cri-docker服务状态sudo systemctl start cri-docker
[kubernetesrenxiaozhao01 ~]$ sudo kubectl apply -f kube-flannel.yml
The connection to the server localhost:8080 was refused - did you specify the right host or port?❓问题五kubeadm join执行失败
❓错误详情一[WARNING Hostname]: hostname renxiaozhao03could not be reached
虽然是警告信息但是名称和IP都不是我之前在/etc/hosts中设置的所以肯定不对对
❗解决方式添加--node-name参数指定主机名
指定--node-name k8s-node02中间加不加都可以不加就用空格
sudo kubeadm join 192.168.17.17:6443 --node-namek8s-node02 --token 4ydg4t.7cjjm52hd4p86gmk --discovery-token-ca-cert-hash sha256:ee2c3ae1c2d702b77a0b52f9dafe734aa7e25f33c44cf7fa469c1adc8c176be1 --cri-socketunix:///var/run/cri-dockerd.sock其中--cri-socketunix:///var/run/cri-dockerd.sock也是根据错误提示新指定的参数其中containerd不知道是不是那个组件自带的博主只装了cri-dockerd这一块也迷糊一方面查资料使用docker容器必须安装cri-dockerd一方面又说只要containerd一步步来吧先跑起来集群再深究
[kubernetesrenxiaozhao02 .kube]$ sudo kubeadm join 192.168.17.17:6443 --node-namek8s-node02 --token 4ydg4t.7cjjm52hd4p86gmk --discovery-token-ca-cert-hash sha256:ee2c3ae1c2d702b77a0b52f9dafe734aa7e25f33c44cf7fa469c1adc8c176be1
Found multiple CRI endpoints on the host. Please define which one do you wish to use by setting the criSocket field in the kubeadm configuration file: unix:///var/run/containerd/containerd.sock, unix:///var/run/cri-dockerd.sock
To see the stack trace of this error execute with --v5 or higher❓错误详情二超时报错error execution phase kubelet-start: error uploading crisocket: nodes k8s-node01 not found
[kubernetesrenxiaozhao02 .kube]$ sudo systemctl stop kubelet
[kubernetesrenxiaozhao02 .kube]$ sudo rm -rf /etc/kubernetes/*
[kubernetesrenxiaozhao02 .kube]$ sudo kubeadm join 192.168.17.17:6443 --token urwtz8.3tvnbe2a63b3fnbl --discovery-token-ca-cert-hash sha256:ee2c3ae1c2d702b77a0b52f9dafe734aa7e25f33c44cf7fa469c1adc8c176be1 --node-name k8s-node01 --cri-socketunix:///var/run/cri-dockerd.sock
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with kubectl -n kube-system get cm kubeadm-config -o yaml
[kubelet-start] Writing kubelet configuration to file /var/lib/kubelet/config.yaml
[kubelet-start] Writing kubelet environment file with flags to file /var/lib/kubelet/kubeadm-flags.env
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
[kubelet-check] Initial timeout of 40s passed.
error execution phase kubelet-start: error uploading crisocket: nodes k8s-node01 not found
To see the stack trace of this error execute with --v5 or higher
[kubernetesrenxiaozhao02 .kube]$
❗解决方式使用kubeadm reset重置之后再添加join
不要通过上面删除的方式博主之前添加成功过renxiaozhao02节点不指定--node-name,默认就是主机名虽然节点已经删除了但是貌似有残留直接在node几点机器执行kubeadm reset然后再添加成功
[kubernetesrenxiaozhao03 ~]$ sudo kubeadm reset --cri-socketunix:///var/run/cri-dockerd.sock
W0830 17:11:12.123313 25230 preflight.go:56] [reset] WARNING: Changes made to this host by kubeadm init or kubeadm join will be reverted.
[reset] Are you sure you want to proceed? [y/N]: y
[preflight] Running pre-flight checks
W0830 17:11:14.355783 25230 removeetcdmember.go:106] [reset] No kubeadm config, using etcd pod spec to get data directory
[reset] Deleted contents of the etcd data directory: /var/lib/etcd
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in /var/lib/kubelet
[reset] Deleting contents of directories: [/etc/kubernetes/manifests /var/lib/kubelet /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.dThe reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually by using the iptables command.If your cluster was setup to utilize IPVS, run ipvsadm --clear (or similar)
to reset your systems IPVS tables.The reset process does not clean your kubeconfig files and you must remove them manually.
Please, check the contents of the $HOME/.kube/config file.
[kubernetesrenxiaozhao03 ~]$ #验证有没有用到ipvsgpt说的咋说呢被它带着走了很远的路
[kubernetesrenxiaozhao03 ~]$ kubectl get svc -A | grep ipvs
[kubernetesrenxiaozhao03 ~]$
[kubernetesrenxiaozhao03 ~]$
[kubernetesrenxiaozhao03 ~]$ #验证有没有用到 CNI
[kubernetesrenxiaozhao03 ~]$ kubectl get svc -A | grep cni -i弯路历程
重新join会报错之前已经生成了相关文件报文件已存在直接执行了删除结果引发新的错sudo systemctl stop kubelet
sudo rm -rf /etc/kubernetes/*直接sudo rm -rf /etc/kubernetes/*结果引发了新的报错/etc/kubernetes/manifests不存在后来网上看了下好像本来就没有这个目录不是删除引起的也不知道之前有没有这个文件貌似只有master节点会用到最终解决方式是直接把mastre节点上的该目录同步过来,或者直接创建mkdir -p /etc/kubernetes/manifests,应该也可以 查看kubelet日志sudo journalctl -u kubelet也被坑了一开始没注意这个命令日志不自动换行改用sudo journalctl -u kubelet --no-pager --followUnable to register node with API server errnodes \k8s-node01\ is forbidden: node \renxiaozhao02\ is not allowed to modify node \k8s-node01\ nodek8s-node01 目录同步过来之后有报错通过sudo journalctl -u kubelet --no-pager --follow可以准确的查看日志8月 30 15:37:47 renxiaozhao02 kubelet[9469]: I0830 15:37:47.153840 9469 kubelet_node_status.go:70] Attempting to register node nodek8s-node01
8月 30 15:37:47 renxiaozhao02 kubelet[9469]: E0830 15:37:47.160006 9469 kubelet_node_status.go:92] Unable to register node with API server errnodes \k8s-node01\ is forbidden: node \renxiaozhao02\ is not allowed to modify node \k8s-node01\ nodek8s-node01
8月 30 15:37:47 renxiaozhao02 kubelet[9469]: I0830 15:37:47.283440 9469 scope.go:117] RemoveContainer containerIDb0ab74aedc14936c3f0e95682f257b3a8981ecb3e9e36b590a35d65c3eafbd16
8月 30 15:37:47 renxiaozhao02 kubelet[9469]: E0830 15:37:47.284025 9469 pod_workers.go:1300] Error syncing pod, skipping errfailed to \StartContainer\ for \kube-scheduler\ with CrashLoopBackOff: \back-off 5m0s restarting failed containerkube-scheduler podkube-scheduler-k8s-node01_kube-system(7734ba1e9e5564ed51f1e93da8155ae7)\ podkube-system/kube-scheduler-k8s-node01 podUID7734ba1e9e5564ed51f1e93da8155ae7
