jenkins+k8s部署集成配置Pipeline agent

[Dominic]

1 k8s部署jenkins

• 1.1 创建jenkins部署空间

kubectl create namespace jenkins

yaml 文件创建参照

https://github.com/jenkinsci/kubernetes-plugin/blob/kubernetes-1.21.5/src/main/kubernetes/jenkins.yml

• 1.2 准备pv&pvc数据持久化存储 volum.yaml

# 创建挂载宿主机目录磁盘
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: standard
  namespace: jenkins
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins-pv-volume
  namespace: jenkins
  labels:
    app: jenkins-storage
    type: local
spec:
  storageClassName: standard
  claimRef:
    name: jenkins-pv-claim
    namespace: jenkins
  capacity:
    storage: 20Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Delete
  local:
    path: /Users/chenqiang/.jenkins
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - docker-desktop # kubernetes默认node节点名称比如minikube
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-pv-claim
  namespace: jenkins
  labels:
    app: jenkins-storage
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 20Gi

• 1.3 创建jenkins Deployment.yaml文件

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins
  labels:
    name: jenkins
  namespace: jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      name: jenkins
  template:
    metadata:
      name: jenkins
      labels:
        name: jenkins
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: jenkins
          image: jenkins/jenkins:lts-alpine-jdk11
          imagePullPolicy: IfNotPresent
          securityContext:
            runAsUser: 0                   #设置以ROOT用户运行容器
            privileged: true
          ports:
            - name: web
              containerPort: 8080
              protocol: TCP
            - name: agent
              containerPort: 50000
              protocol: TCP
          resources:
            limits:
              cpu: "1000m"
              memory: 2Gi
            requests:
              cpu: "500m"
              memory: 1Gi
          env:
            - name: LIMITS_MEMORY
              valueFrom:
                resourceFieldRef:
                  resource: limits.memory
                  divisor: 1Mi
            - name: JAVA_OPTS
              value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
          livenessProbe:
            httpGet:
              path: "/login"
              port: 8080
            initialDelaySeconds: 90
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 12
          readinessProbe:
            httpGet:
              path: "/login"
              port: 8080
            initialDelaySeconds: 60
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 12
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/jenkins_home
            - name: system
              mountPath: /etc/profile
            - name: docker
              mountPath: /var/run/docker.sock
            - name: docker-home
              mountPath: /bin/docker
            - name: users-home
              mountPath: /root
      volumes:
        - name: jenkins-home
          persistentVolumeClaim: 
            claimName: jenkins-pv-claim
        - name: docker
          hostPath:
            path: /var/run/docker.sock
            type: ""
        - name: docker-home
          hostPath:
            path: /bin/docker
            type: ""
        - name: system
          hostPath:
            path: /etc/profile
              # 宿主机的环境变量
        - name: users-home
          hostPath:
            path: /Users/chenqiang

• 1.4 创建jenkins服务service.yaml文件

---
apiVersion: v1
kind: Service
metadata:
  name: jenkins
  namespace: jenkins
spec:
  selector:
    name: jenkins
  type: NodePort
  ports:
    - name: web
      port: 8080
      targetPort: web
      protocol: TCP
      nodePort: 30008
    - name: agent
      port: 50000
      targetPort: agent
      protocol: TCP

1.5 创建nginx反向代理ingress.yaml

# 若不走反向代理这一步可省略
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: jenkins
  namespace: jenkins
  labels:
    name: jenkins
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: 50m
    nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
    ingress.kubernetes.io/ssl-redirect: "true"
    ingress.kubernetes.io/proxy-body-size: 50m
    ingress.kubernetes.io/proxy-request-buffering: "off"

spec:
  ingressClassName: nginx
  rules:
  - host: k8s.jenkins.com 
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: jenkins
            port:
              number: 8080
  tls:
  - hosts:
    - k8s.jenkins.com
    secretName: lts-jenkins

• 1.6 修改/etc/hosts 做域名映射

sudo vi /etc/hosts

2 配置kubernetes节点

• 2.1 安装jenkins 插件 kubernetes
  
  
• 2.2 创建kubernetes节点
  Dashboard → Manage Jenkins → Clouds → 十 New cloud
  
  
• 2.3 配置节点
  • Name: 配置的名称
  • Kubernetes URL: 这里的URL是K8s内部的URL，实际上就是svcname https://kubernetes.default.svc.cluster.local
  • Kubernetes Namespace: 是Jenkins所在的命名空间namespace）

• Jenkins URL: URL是jenkins的svc名称加上命名空间，也可以是 svc 的CLUSTER-IP:PORT
  http://jenkins.jenkins.svc.cluster.local:8080
  http://10.107.183.199:8080

• ** 2.4 新建 Pod templates**
  Dashboard → Manage Jenkins → Clouds → kubernetes → + Add a pod template
  
  
• 2.5 配置 Podtemplates
  • 命名
    
    
    
    
    
    
    
    

3 创建Pipeline任务测试

def label = "jenkins-slave"
podTemplate(label: label, cloud: 'kubernetes')
{
node(label) {
        stage('pull code') {
            echo "拉取代码"
        }
        stage('build') {
            echo "代码编译"
        }
        stage('SonarQube') {
            echo "质量扫描"
        }
    }
}

参考官方文档地址

https://plugins.jenkins.io/kubernetes/