网站建设和管理经验,北京网站优化专家,国外域名注册支持支付宝,西安网站改版的公司前言
在持续集成和部署中#xff0c;我们通常需要部署多个实例或组件到Kubernetes集群中。通过Jenkins的管道脚本#xff0c;我们可以自动化这个过程。在本文中#xff0c;我将演示如何使用Jenkins Pipeline及单个YAML模板文件#xff08;.tpl#xff09;来部署多个类似的…前言
在持续集成和部署中我们通常需要部署多个实例或组件到Kubernetes集群中。通过Jenkins的管道脚本我们可以自动化这个过程。在本文中我将演示如何使用Jenkins Pipeline及单个YAML模板文件.tpl来部署多个类似的Kubernetes组件而不需要为每个组件提供单独的模板文件。
问题背景
参照Jenkins Pipeline 脚本优化实践从繁琐到简洁 批量生成 Kubernetes 部署模板从 1 到20顺序模板
pipeline {agent none // Use none at the top level, each stage will define its own agent.environment {REGISTRY xxxx/master-metaspaceKUBE_CONFIG --namespacemaster-metaspace --contextmasterKUBE_YAML_PATH /home/jenkins/workspace/yaml/master-metaspace// Assume that data is defined elsewhere or injected as a parameter.BASE_WORKSPACE xxxxxxx // 定义一个基础工作空间路径}stages {stage(GetCode) {agent { label build01 }steps {script {checkout scm: [$class: GitSCM,branches: [[name: env.branchName]],extensions: [[$class: CloneOption, depth: 1, noTags: false, shallow: true]],userRemoteConfigs: [[credentialsId: xxxx, url: env.gitHttpURL]]]}}}stage(Docker Builds) {parallel {stage(Build dataloader-game-ucenter) {agent { label build01 }when { environment name: dataloader, value: true }steps {buildAndPushDockerImage(dataloader-game-ucenter, env.data, env.BASE_WORKSPACE)}}stage(Build datawriter-game-ucenter) {agent { label build01 }when { environment name: datawriter, value: true }steps {buildAndPushDockerImage(datawriter-game-ucenter, env.data, env.BASE_WORKSPACE)}}stage(Build game-ucenter) {agent { label build01 }when { environment name: game-ucenter, value: true }steps {buildAndPushDockerImage(game-ucenter, env.data, env.BASE_WORKSPACE)}}}}stage(Development Deployment) {parallel {stage(Deploy datawriter-game-ucenter) {when { environment name: datawriter-game-ucenter, value: true }agent { label huaweiyun-xx }steps {deployToKubernetes(datawriter-game-ucenter)}}stage(Deploy dataloader-game-ucenter) {when { environment name: dataloader, value: true }agent { label huaweiyun-xx }steps {deployToKubernetes(dataloader-game-ucenter)}}stage(Deploy game-ucenter) {when { environment name: game-ucenter, value: true }agent { label huaweiyun-xx }steps {deployToKubernetes(game-ucenter-1)deployToKubernetes(game-ucenter-2)deployToKubernetes(game-ucenter-3)deployToKubernetes(game-ucenter-4)............................}}}}}
}// Define methods outside pipeline to avoid repetitiondef buildAndPushDockerImage(String imageName, String tag, String workspacePath) {sh cd ${workspacePath} echo Current directory: \$(pwd) // 使用基础工作空间变量sh cd ${workspacePath}/${imageName} docker build --build-arg NODE_ENV$imageName -t $REGISTRY/$imageName:$tag .withCredentials([usernamePassword(credentialsId: xxxxx, passwordVariable: dockerPassword, usernameVariable: dockerUser)]) {sh docker login -u $dockerUser -p $dockerPassword $REGISTRYsh docker push $REGISTRY/$imageName:$tag}
}def deployToKubernetes(String kubernetesComponent) {String templateFile ${KUBE_YAML_PATH}/${kubernetesComponent}.tplString outputFile ${KUBE_YAML_PATH}/${kubernetesComponent}.yamlsh sed -e s/{data}/$data/g $templateFile $outputFilesh sudo kubectl apply -f $outputFile $KUBE_CONFIG
}默认jenkins pipeline如上我们有多个相似的游戏用户中心服务game-ucenter-*运行在Kubernetes集群中它们都使用非常相似的Kubernetes YAML配置文件配置文件之间的差异主要是一些标识符的不同例如服务的序号。在传统的做法中维护一系列几乎一样的模板文件如game-ucenter-1.tpl, game-ucenter-2.tpl 等将非常低效且易出错。
为了精简流程和提高效率我们需要一个方法来通过单一模板生成多个配置文件并由此部署多个不同的服务实例。
解决方案
使用Jenkins Pipeline中的sed命令和循环结构我们可以从单一模板生成多个Kubernetes配置文件并相应地部署每个服务实例。参照generate_templates.sh脚本
#!/bin/bash# Define the name of the template file.
TEMPLATE_FILEgame-ucenter.tpl# Check if the template file exists.
if [ ! -f $TEMPLATE_FILE ]; thenecho Template file $TEMPLATE_FILE does not exist.exit 1
fi# Loop to create files from game-ucenter-2 to pvp-game-20 based on the template.
for i in $(seq 1 20); do# Define the name of the new file.NEW_FILEgame-ucenter-${i}.yaml# Copy the template to the new file.cp $TEMPLATE_FILE $NEW_FILE# Use sed to replace game-ucenter-1 with game-ucenter-N and save inline (-i option).sed -i s/game-ucenter/game-ucenter-${i}/g $NEW_FILEecho Created file: $NEW_FILE
doneecho All files created successfully.
步骤 1: 定义Jenkins Pipeline
在我们的Jenkins脚本中我们首先定义了基础环境变量和两个函数buildAndPushDockerImage 和 deployToKubernetes。这些函数将用于构建Docker镜像并部署到Kubernetes
def buildAndPushDockerImage(String imageName, String tag, String workspacePath) {sh cd ${workspacePath} echo Current directory: \$(pwd) // 使用基础工作空间变量sh cd ${workspacePath}/${imageName} docker build --build-arg NODE_ENV$imageName -t $REGISTRY/$imageName:$tag .withCredentials([usernamePassword(credentialsId: xxx, passwordVariable: dockerPassword, usernameVariable: dockerUser)]) {sh docker login -u $dockerUser -p $dockerPassword $REGISTRYsh docker push $REGISTRY/$imageName:$tag}
}def deployToKubernetes(String kubernetesComponent) {String templateFile ${KUBE_YAML_PATH}/${kubernetesComponent}.tplString outputFile ${KUBE_YAML_PATH}/${kubernetesComponent}.yamlsh sed -e s/{data}/$data/g $templateFile $outputFilesh sudo kubectl apply -f $outputFile $KUBE_CONFIG
}步骤 2: 修改deployToKubernetes函数
接下来我们需要修改deployToKubernetes函数以便它能够接受组件名称并使用单一模板文件创建具体的配置文件。
def deployToKubernetes(String kubernetesComponent, int instance1, int totalInstances1) {// 检查实例值if (instance 1) {error(实例数必须大于0)}// 根据 instance 的值来定义资源的名称和文件名String nameSuffix totalInstances 1 ? -${instance} : // 总是添加后缀除非只有一个实例String outputFileName ${kubernetesComponent}${nameSuffix}.yamlString templateFile ${KUBE_YAML_PATH}/${kubernetesComponent}.tplString outputFile ${KUBE_YAML_PATH}/${outputFileName}String nameReplacement ${kubernetesComponent}${nameSuffix}sh cat ${templateFile} \| sed s/{data}/${data}/g \| sed s/name: ${kubernetesComponent}/name: ${nameReplacement}/g \| sed s/app: ${kubernetesComponent}/app: ${nameReplacement}/g \ ${outputFile}// 使用 KUBE_CONFIG 应用 Kubernetes 配置sh kubectl apply -f ${outputFile} ${KUBE_CONFIG}
}对于单实例的业务例如Deploy dataloader-game-ucenter我们不需要传递实例编号。
stage(Deploy dataloader-game-ucenter) {when { environment name: dataloader, value: true }agent { label huaweiyun-xx }steps {deployToKubernetes(dataloader-game-ucenter)}
}对于多实例。我这里生成 规则优点强迫症了。如果多实例我生成的规则要求符合game-ucenter-1,game-ucenter-2,game-ucenter-3…顺序当单个实例的时候则保持原来的不加标签 stage(Deploy game-ucenter) {when { environment name: game-ucenter, value: true }agent { label k8s-node-06 }steps {script {int instances 2 // 假设我们有2个实例for (int i 1; i instances; i) {def componentName game-ucenterdeployToKubernetes(game-ucenter, i, instances)}}}}步骤 3: 准备模板文件
我们的模板文件game-ucenter.tpl将包含通用的Kubernetes服务或部署定义使用占位符game-ucenter-1 game-ucenter-2来标识应该被替换的地方。
# game-ucenter-1.tpl (示例部分)
apiVersion: apps/v1
kind: Deployment
metadata:name: game-ucenter
spec:replicas: 1strategy:rollingUpdate:maxSurge: 1maxUnavailable: 0selector:matchLabels:app: game-ucentertemplate:metadata:labels:app: game-ucenterspec:containers:- name: game-ucenterimage: xxxx/xxx/game-ucenter:{data}envFrom:- configMapRef:name: deployports:- containerPort: 80resources:requests:memory: 4096Mcpu: 2000mlimits:memory: 4096Mcpu: 2000m livenessProbe:httpGet:scheme: HTTPpath: /test.htmlport: 80initialDelaySeconds: 20periodSeconds: 120successThreshold: 1failureThreshold: 3readinessProbe:httpGet:scheme: HTTPpath: /test.htmlport: 80initialDelaySeconds: 20periodSeconds: 120imagePullSecrets: - name: xxx
---apiVersion: v1
kind: Service
metadata:name: game-ucenterlabels:app: game-ucenter
spec:ports:- port: 80protocol: TCPtargetPort: 80selector:app: game-ucenter# ...步骤 4: 执行Jenkins Pipeline
当Jenkins Pipeline运行到Development Deployment阶段时它将循环创建和应用game-ucenter-1.yaml到game-ucenter-2.yaml的配置文件从而部署2个game-ucenterdeployment服务实例。 并保证单个实例的原有命名规则
通过这一方法我们不再需要为每个服务实例维护一个单独的模板文件而是可以通过一个模板文件和Jenkins Pipeline的自动化来简化服务部署工作。这样做不仅提升了效率也降低了出错的风险。
注意
以上代码和命令为示例性质可能需要根据您具体的Jenkins环境和Kubernetes集群进行相应的调整。在生产环境中部署之前请确保进行充分的测试。
