用rem做移动网站,现在不流行做网站了么,四川教育公共信息服务平台,茶叶价格网站建设目录
一、前期准备
二、安装前配置
三、安装docker
四、安装cri-dockerd
五、部署k8s master节点
六、整合kubectl与cri-dockerd
七、网络等插件安装
八、常见问题及解决方法 一、前期准备
①ubuntu系统 本地已安装ubuntu系统#xff0c;lsb_release -a命令查看版本信…目录
一、前期准备
二、安装前配置
三、安装docker
四、安装cri-dockerd
五、部署k8s master节点
六、整合kubectl与cri-dockerd
七、网络等插件安装
八、常见问题及解决方法 一、前期准备
①ubuntu系统 本地已安装ubuntu系统lsb_release -a命令查看版本信息 ②安装包
相关资源也可以从这里下载
https://download.csdn.net/download/qq_41061437/88806777
https://download.csdn.net/download/qq_41061437/88806805
https://download.csdn.net/download/qq_41061437/88806805
国内很多镜像可以在这里找docker-ce-linux安装包下载_开源镜像站-阿里云docker-ce-linux安装包是阿里云官方提供的开源镜像免费下载服务每天下载量过亿阿里巴巴开源镜像站为包含docker-ce-linux安装包的几百个操作系统镜像和依赖包镜像进行免费CDN加速更新频率高、稳定安全。https://mirrors.aliyun.com/docker-ce/linux/?spma2c6h.25603864.0.0.5a9461d5d8WvU6 docker containerd.io_1.6.22-1_amd64.deb 下载地址Index of linux/ubuntu/dists/focal/pool/stable/amd64/https://download.docker.com/linux/ubuntu/dists/focal/pool/stable/amd64/docker.io_24.0.5-0ubuntu1_20.04.1_amd64.deb 下载地址 https://ubuntu.pkgs.org/20.04/ubuntu-updates-universe-amd64/docker.io_24.0.5-0ubuntu1~20.04.1_amd64.deb.htmlhttps://ubuntu.pkgs.org/20.04/ubuntu-updates-universe-amd64/docker.io_24.0.5-0ubuntu1~20.04.1_amd64.deb.htmlcri-dockerd cri-dockerd_0.3.4.3-0.ubuntu-jammy_amd64.deb 下载地址 Releases · Mirantis/cri-dockerd · GitHubdockerd as a compliant Container Runtime Interface for Kubernetes - Releases · Mirantis/cri-dockerdhttps://github.com/Mirantis/cri-dockerd/releasesk8s kubeadm_1.27.4-00_amd64.deb kubectl_1.27.4-00_amd64.deb kubelet_1.27.4-00_amd64.deb 下载地址 Index of /kubernetes/apt/pool/http://mirrors.ustc.edu.cn/kubernetes/apt/pool/ 其它 ipset_7.5-1ubuntu0.20.04.1_amd64.deb https://ubuntu.pkgs.org/20.04/ubuntu-main-arm64/ipset_7.5-1~exp1_arm64.deb.htmlhttps://ubuntu.pkgs.org/20.04/ubuntu-main-arm64/ipset_7.5-1~exp1_arm64.deb.htmlipvsadm_1:1.31-1_amd64.deb ipvsadm package : Ubuntuhttps://launchpad.net/ubuntu/source/ipvsadmlibipset13_7.5-1ubuntu0.20.04.1_amd64.deb 7.5-1ubuntu0.20.04.1 : libipset13 : amd64 : Focal (20.04) : UbuntuIP sets are a framework inside the Linux 2.4.x and 2.6.x kernel which can be administered by the ipset(8) utility. Depending on the type, currently an IP set may store IP addresses, (TCP/UDP) port numbers or IP addresses with MAC addresses in a way which ensures lightning speed when matching an entry against a set. . If you want to . * store multiple IP addresses or port numbers and match against the entire collection using a single iptables rule. * dynamically update iptable...https://bugs.launchpad.net/ubuntu/focal/amd64/libipset13/7.5-1ubuntu0.20.04.1kubernetes-cni_1.2.0-00_s390x_86cdf4d82e3a59c3f6e12975b149a5e42afebff3fd342161abac520253237938.deb cri-tools_1.26.0-00_amd64.de Index of /kubernetes/apt/pool/http://mirrors.ustc.edu.cn/kubernetes/apt/pool/socat_1.7.3.3-2_amd64.deb 1.7.3.3-2 : socat : amd64 : Focal (20.04) : UbuntuSocat (for SOcket CAT) establishes two bidirectional byte streams and transfers data between them. Data channels may be files, pipes, devices (terminal or modem, etc.), or sockets (Unix, IPv4, IPv6, raw, UDP, TCP, SSL). It provides forking, logging and tracing, different modes for interprocess communication and many more options. . It can be used, for example, as a TCP relay (one-shot or daemon), as an external socksifier, as a shell interface to Unix sockets, as an IPv6 relay, as a ...https://launchpad.net/ubuntu/focal/amd64/socat/1.7.3.3-2ebtables_2.0.11-3build1_amd64.deb https://ubuntu.pkgs.org/20.04/ubuntu-main-amd64/ebtables_2.0.11-3build1_amd64.deb.htmlhttps://ubuntu.pkgs.org/20.04/ubuntu-main-amd64/ebtables_2.0.11-3build1_amd64.deb.html conntrack_1%3A1.4.5-2_amd64.deb https://ubuntu.pkgs.org/20.04/ubuntu-main-amd64/conntrack_1.4.5-2_amd64.deb.htmlhttps://ubuntu.pkgs.org/20.04/ubuntu-main-amd64/conntrack_1.4.5-2_amd64.deb.html 二、安装前配置
①设置主机名hostname管理节点设置主机名为master # 需要设置其他主机名称时可将 master 替换为正确的主机名node1、node2即可。sudo hostnamectl set-hostname master ②编辑/etc/hosts 文件添加域名解析 sudo vim /etc/hosts cat EOF /etc/hosts10.10.10.10 masterEOF ③关闭防火墙、selinux和swap sudo systemctl stop firewalldsudo systemctl disable firewalldsudo setenforce 0sudo sed -i s/^SELINUXenforcing/SELINUXdisabled/g /etc/selinux/configsudo swapoff -a ④禁用selinux sudo apt-get install selinux-utils sudo apt --fix-broken install sudo setenforce 0 ⑤禁用swap分区 sudo swapoff -a 禁用后如需开启swap 1、vim /etc/default/kubelet #添加内容 KUBELET_EXTRA_ARGS--fail-swap-onfalse 2、sudo vim /var/lib/kubelet/config.yaml #请修改或追加以下内容 featureGates: NodeSwap: true memorySwap: swapBehavior: UnlimitedSwap 3、sudo systemctl start kubelet ⑥关闭防火墙 查看当前的防火墙状态sudo ufw status 关闭防火墙: sudo ufw disable ⑦设置服务启动参数 sudo vim /etc/sysctl.d/k8s.conf 参数内容如下 net.bridge.bridge-nf-call-iptables1 net.bridge.bridge-nf-call-ip6tables1 net.ipv4.ip_forward1 vm.swappiness0 vm.overcommit_memory1 vm.panic_on_oom0 fs.inotify.max_user_instances8192 fs.inotify.max_user_watches1048576 fs.file-max52706963 fs.nr_open52706963 net.ipv6.conf.all.disable_ipv61 net.netfilter.nf_conntrack_max2310720 使配置生效 sudo sysctl -p /etc/sysctl.d/k8s.conf 如果执行报错见八①
⑧安装ipvs内核模块
由于ubuntu系统默认已经加载ipvs内核模块执行验证 lsmod | grep ip_vs 如果返回 为空表示没有加载执行下面命令重新加载 sudo modprobe ip_vs 安装 ipvsadm ipset sudo dpkg -i ipset_7.5-1ubuntu0.20.04.1_amd64.deb sudo dpkg -i ipvsadm_1:1.31-1_amd64.deb sudo dpkg -i libipset13_7.5-1ubuntu0.20.04.1_amd64.deb 三、安装docker
①需要用到的离线包
containerd.io_1.6.22-1_amd64.deb
docker.io_24.0.5-0ubuntu1_20.04.1_amd64.deb
执行命令安装 sudo dpkg -i containerd.io_1.6.22-1_amd64.deb sudo dpkg -i docker.io_24.0.5-0ubuntu1_20.04.1_amd64.deb sudo systemctl start containerd sudo systemctl start docker 查看状态 sudo systemctl status docker ②配置用户组
把需要使用docker命令的用户添加到用户组中 sudo groupadd docker sudo usermod -aG docker $USER $USER是环境变量指当前用户
配置docker开机启动 sudo systemctl enable docker 如果配置开机启动报错见八②
③配置私有镜像仓库以及日志切分配置
修改daemon.json文件 sudo vim /etc/docker/daemon.json { registry-mirrors: [ https://registry.docker-cn.com ], insecure-registry: [ registry.docker-cn.com ], log-driver:json-file, log-opts:{ max-size:100m, max-file:3 } } 四、安装cri-dockerd
cri-docker是一个支持CRI标准的shim垫片一边通过CRI跟kubelet交互另一边跟docker api交互从而间接的实现了kubernetes以docker作为容器运行。
需要用到的离线包cri-dockerd_0.3.4.3-0.ubuntu-jammy_amd64.deb
执行命令 sudo dpkg -i cri-dockerd_0.3.4.3-0.ubuntu-jammy_amd64.deb 安装完cri-docker后对应的服务会自动启动命令查看 systemctl status cri-docker 五、部署k8s master节点
①需要用到的离线包 kubeadm_1.27.4-00_amd64.deb kubelet_1.27.4-00_amd64.deb kubectl_1.27.4-00_amd64.deb 执行命令 sudo dpkg -i kubeadm_1.27.4-00_amd64.deb sudo dpkg -i kubelet_1.27.4-00_amd64.deb sudo dpkg -i kubectl_1.27.4-00_amd64.deb 如果报错见八③
导入k8s集群安装所需镜像 docker load -i k8s_images.tar 验证镜像是否存在 docker image ls 六、整合kubectl与cri-dockerd
①修改配置文件 sudo vim /lib/systemd/system/cri-docker.service 内容如下 ExecStart/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugincni --cni-bin-dir/opt/cni/bin --cni-cache-dir/var/lib/cni/cache --cni-conf-dir/etc/cni/net.d --pod-infra-container-imageregistry.aliyuncs.com/google_containers/pause:3.9 修改后的cri-docker.service文件内容 [Unit] DescriptionCRI Interface for Docker Application Container Engine Documentationhttps://docs.mirantis.com Afternetwork-online.target firewalld.service docker.service Wantsnetwork-online.target Requirescri-docker.socket [Service] Typenotify ExecStart/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugincni --cni-bin-dir/opt/cni/bin --cni-cache-dir/var/lib/cni/cache --cni-conf-dir/etc/cni/net.d --pod-infra-container-imageregistry.aliyuncs.com/google_containers/pause:3.9 ExecReload/bin/kill -s HUP $MAINPID TimeoutSec0 RestartSec2 Restartalways # Note that StartLimit* options were moved from Service to Unit in systemd 229. # Both the old, and new location are accepted by systemd 229 and up, so using the old location # to make them work for either version of systemd. StartLimitBurst3 # Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230. # Both the old, and new name are accepted by systemd 230 and up, so using the old name to make # this option work for either version of systemd. StartLimitInterval60s # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILEinfinity LimitNPROCinfinity LimitCOREinfinity # Comment TasksMax if your systemd version does not support it. # Only systemd 226 and above support this option. TasksMaxinfinity Delegateyes KillModeprocess [Install] WantedBymulti-user.target 在/usr/lib/systemd/system/cri-docker.service文件中添加上如上配置 --network-plugin指定网络插件规范的类型这里要使用CNI --cni-bin-dir指定CNI插件二进制程序文件的搜索目录 --cni-cache-dirCNI插件使用的缓存目录 --cni-conf-dirCNI插件加载配置文件的目录 --pod-infra-container-image:指定pause镜像 这个一定要配置不然systemctl status cri-docker会报错 ②重启cri-dockerd服务 sudo systemctl daemon-reload sudo systemctl restart cri-docker 七、网络等插件安装
①kubeadm初始化 sudo kubeadm init --kubernetes-versionv1.27.4 --apiserver-advertise-addressxxxxxx --apiserver-bind-port6443 --image-repositoryregistry.aliyuncs.com/google_containers --service-cidr10.96.0.0/12 --pod-network-cidr10.244.0.0/16 --ignore-preflight-errorsSwap --cri-socketunix:///run/cri-dockerd.sock --v5 其中piserver-advertise-address是实际机器的ip地址
如果报错见八④
②配置kubectl工具
root用户 sudo mkdir -p /root/.kubesudo cp /etc/kubernetes/admin.conf /root/.kube/config 普通用户 ##创建自己的kubemkdir -p $HOME/.kube##复制root用户的kubectl配置到家目录下sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config##修改kubectl配置文件的权限sudo chown $(id -u):$(id -g) $HOME/.kube/config 测试集群 kubectl get nodes kubectl get cs kubectl get pods --all-namespaces 我这里是已经部署完毕并已经成功运行pod后的截图 ③安装calico
需要用到calico.yaml,详见
https://docs.projectcalico.org/v3.20/manifests/calico.yamlhttps://docs.projectcalico.org/v3.20/manifests/calico.yaml执行命令 kubectl apply -f calico.yml 部署完毕之后coredns的两个pod变为runing状态
执行命令查看 kubectl get pod -n kube-system ④设置允许master调度pod
先执行命令查看 sudo kubectl describe node master | grep Taints 会显示 Taints: node-role.kubernetes.io/control-plane:NoSchedule 执行命令 kubectl taint node master node-role.kubernetes.io/control-plane:NoSchedule- 如果执行kubectl get nodes出现node没有ready重启container和kubectl systemctl restart containerd systemctl restart kubelet 如果报错见八⑤
八、常见问题及解决方法
①sysctl: cannot stat /proc/sys/net/bridge/bridge-nf-call-ip6tables: No Such file or directory 如果出现这个错误说明没有先加载内核模块br_netfilterbridge-nf 使 netfilter 可以对 Linux 网桥上的 IPv4/ARP/IPv6 包过滤。比如设置net.bridge.bridge-nf-call-iptables1后二层的网桥在转发包时也会被 iptables的 FORWARD 规则所过滤。
解决方案 1.加载overlay和br_netfilter两个内核模块 sudo modprobe overlay sudo modprobe br_netfilter 持久化加载上述两个模块避免重启失效。 $ cat EOF | sudo tee /etc/modules-load.d/k8s.conf overlay br_netfilter EOF 通过运行 lsmod | grep br_netfilter 来验证 br_netfilter 模块是否已加载 通过运行 lsmod | grep overlay 来验证 overlay模块是否已加载 再次执行sudo sysctl -p /etc/sysctl.d/k8s.conf 如果报错sysctl: cannot stat /proc/sys/net/netfilter/nf_conntrack_max: No such file or directory可能是 conntrack没有加载执行lsmod | grep conntrack 如果返回为空表示没有加载执行下面命令 重新加载sudo modprobe ip_conntrack sudo sysctl -p /etc/sysctl.d/k8s.conf ②配置docker开机启动报错Failed to enable unit: Unit file docker.service does not exist.
新建docker.service文件sudo vim /lib/systemd/system/docker.service [Unit] DescriptionDocker Application Container Engine Documentationhttps://docs.docker.com Afternetwork-online.target docker.socket firewalld.service containerd.service time-set.target Wantsnetwork-online.target containerd.service Requiresdocker.socket [Service] Typenotify # the default is not to use systemd for cgroups because the delegate issues still # exists and systemd currently does not support the cgroup feature set required # for containers run by docker ExecStart/usr/bin/dockerd -H fd:// --containerd/run/containerd/containerd.sock ExecReload/bin/kill -s HUP $MAINPID TimeoutStartSec0 RestartSec2 Restartalways # Note that StartLimit* options were moved from Service to Unit in systemd 229. # Both the old, and new location are accepted by systemd 229 and up, so using the old location # to make them work for either version of systemd. StartLimitBurst3 # Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230. # Both the old, and new name are accepted by systemd 230 and up, so using the old name to make # this option work for either version of systemd. StartLimitInterval60s # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILEinfinity LimitNPROCinfinity LimitCOREinfinity # Comment TasksMax if your systemd version does not support it. # Only systemd 226 and above support this option. TasksMaxinfinity # set delegate yes so that systemd does not reset the cgroups of docker containers Delegateyes # kill only the docker process, not all processes in the cgroup KillModeprocess OOMScoreAdjust-500 [Install] WantedBymulti-user.target ③安装kubeadm、kubelet、kubectl报错
如果报错 Selecting previously unselected package kubeadm. (Reading database ... 106973 files and directories currently installed.) Preparing to unpack kubeadm_1.27.4-00_amd64.deb ... Unpacking kubeadm (1.27.4-00) ... Selecting previously unselected package kubelet. Preparing to unpack kubelet_1.27.4-00_amd64.deb ... Unpacking kubelet (1.27.4-00) ... Selecting previously unselected package kubectl. Preparing to unpack kubectl_1.27.4-00_amd64.deb ... Unpacking kubectl (1.27.4-00) ... dpkg: dependency problems prevent configuration of kubeadm: kubeadm depends on kubernetes-cni ( 1.1.1); however: Package kubernetes-cni is not installed. kubeadm depends on cri-tools ( 1.25.0); however: Package cri-tools is not installed. dpkg: error processing package kubeadm (--install): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of kubelet: kubelet depends on kubernetes-cni ( 1.1.1); however: Package kubernetes-cni is not installed. kubelet depends on socat; however: Package socat is not installed. kubelet depends on ebtables; however: Package ebtables is not installed. kubelet depends on conntrack; however: Package conntrack is not installed. dpkg: error processing package kubelet (--install): dependency problems - leaving unconfigured Setting up kubectl (1.27.4-00) ... Errors were encountered while processing: kubeadm kubelet 需要安装依赖kubernetes-cni cri-tools socat ebtables conntrack sudo dpkg -i kubernetes-cni_1.2.0-00_s390x_86cdf4d82e3a59c3f6e12975b149a5e42afebff3fd342161abac520253237938.deb sudo dpkg -i cri-tools_1.26.0-00_amd64.deb sudo dpkg -i socat_1.7.3.3-2_amd64.deb sudo dpkg -i ebtables_2.0.11-3build1_amd64.deb sudo dpkg -i conntrack_1%3A1.4.5-2_amd64.deb 如果安装继续报错 dpkg: error processing archive kubernetes-cni_1.2.0-00_arm64_5d61b8d04701612640667c1da13b616529ded1fed0b7405382d8d08eaa5b5af7.deb (--install): package architecture (arm64) does not match system (amd64) Errors were encountered while processing: kubernetes-cni_1.2.0-00_arm64_5d61b8d04701612640667c1da13b616529ded1fed0b7405382d8d08eaa5b5af7.deb halosbgi:/mnt/test-halos/upgradeworkspace/deb$ sudo dpkg --add-architecture arm64 再次安装kubernetes-cni_1.2.0-00_arm64_5d61b8d04701612640667c1da13b616529ded1fed0b7405382d8d08eaa5b5af7.deb sudo dpkg -i cri-tools_1.26.0-00_arm64_be3fa6bdc17ab229b45222887c442ae1a601b3b2bc3e011c9e7235767e7269c4.deb ④kubeadm初始化报错
如果报错 [ERROR Port-10250]: Port 10250 is in use 要解除kubelet.service的mask状态请使用以下命令 sudo systemctl unmask kubelet.service 驱动一致还报错执行以下命令 sudo kubeadm reset -f sudo iptables -F sudo iptables -t nat -F sudo iptables -t mangle -F sudo iptables -X 如果继续报错 [init] Using Kubernetes version: v1.24.4 [preflight] Running pre-flight checks error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR CRI]: container runtime is not running: output: E1107 11:17:24.937456 31126 remote_runtime.go:948] Status from runtime service failed errrpc error: code Unimplemented desc unknown service runtime.v1alpha2.RuntimeService time2022-11-07T11:17:2408:00 levelfatal msggetting status of runtime: rpc error: code Unimplemented desc unknown service runtime.v1alpha2.RuntimeService , error: exit status 1 [preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors... To see the stack trace of this error execute with --v5 or higher 执行以下命令 sudo rm-rf /etc/containerd/config.toml sudo systemctl daemon-reload sudo systemctl restart containerd 执行完之后再重新执行kubeadm init命令
⑤设置允许master调度pod报错 kubectl taint node master node-role.kubernetes.io/control-plane:NoSchedule- error: taint node-role.kubernetes.io/control-plane:NoSchedule not found 出现该错误会导致pod节点无法调度命令df -h查看系统磁盘空间我这里的问题主要就是磁盘空间不足导致。
