Git+Jenkins+Harbor+Docker实现CICD一些记录

发表于 | 分类于 | 阅读次数:

update: ‘2021/8/22’

一. 环境准备:

ctntos8.4 cpu:4 内存: 2048M 硬盘: 256G

192.168.1.102 Harbor

192.168.1.103 Jenkins

192.168.1.104 Docker

image-20210822021127881

二. 一些问题:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# sshd
systemctl start sshd
ps -e | grep sshd

# 修改hostname
vi /etc/hostname
hostnamectl set-hostname appjzw

# ping 不通外网
vim /etc/resolv.conf

nameserver 8.8.8.8
nameserver 202.106.0.20

nmcli c reload enp0s3

# centos8 yum 换源
# 1.备份旧的源
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
# 2.下载对应版本的阿里源（wget -O **下载并以指定的文件名保存**）
wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-8.repo
yum makecache # 3.删除yum旧的缓存，生成新的换源后的缓存
cat /etc/yum.repos.d/CentOS-Base.repo # 4.检查是否换源成功

三. 部署Harbor镜像仓库

1.下载安装

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
wget [https://github.com/goharbor/harbor/releases/download/v2.3.1/harbor-offline-installer-v2.3.1.tgz](https://github.com/goharbor/harbor/releases/download/v2.3.1/harbor-offline-installer-v2.3.1.tgz)
yum install lrzsz

# 从服务器拉东西
scp root@[公网地址]:/root/harbor-fooline-installer-v2.3.1.tgz /usr/yzhou/Desktop

systemctl start docker
docker -v
docker —version
docker info

docker images
docker ps

cp harbor.yml.tmpl harbor.yml
vim harbor.yml

./install.sh

cd harbor
docker-compose restart
docker-compose down -v
docker-compose up -d

2. 测试Harbor

1
2
3
# name
# http登录 
# port

image-20210821230929660

3. Dockerfile文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# cd /root/jenkins/docker-file/maven-docker-test_war
# vim Dockerfile
# Version 1.0
# Base images.
FROM tomcat:8.0.36-alpine

# Author.
MAINTAINER yzhou<403708416@qq.com>

# Add war.
ADD maven-docker.war /usr/local/tomcat/webapps/

# Define working directory.
ENV PATH /usr/local/tomcat/bin:$PATH

# Define environment variavles.
ENV PATH /usr/loacl/tomcat/bin:$PATH

# Define default command.
CMD {"catalina.sh", "run"}

# Expose ports.
EXPOSE 8080

4. Harbor权限相关

image-20210821230456126

四. 业务服务器

1.安装软件

1
2
3
4
5
6
7
yum install -y yum-utils  device-mapper-persistent-data lvm2
docker -v
yum-config-manager  --add-repo   [https://download.docker.com/linux/centos/docker-ce.repo](https://download.docker.com/linux/centos/docker-ce.repo)
yum install docker-ce docker-ce-cli [containerd.io](http://containerd.io/) --nobest
yum install container-selinux
systemctl start docker
yum install jq -y

2.预先配置

1
2
3
4
5
6
7
8
# 在业务服务器上配置:(否则没有权限报错)
[SSH] executing...
sudo: sorry, you must have a tty to run sudo
docker: invalid reference format.
# visudo
#
# Defaults requiretty
Defaults:root !requiretty

五. jenkins

1. 问题:

1
2
3
4
5
6
7
8
9
# 登录harbor遇到问题
docker login -u admin -p Harbor12345 reg.mydomain.com
Error response from daemon: Get https://reg.mydomain.com/v2/: unauthorized: authentication required
# vim /etc/docker/daemon.json
{
  "registry-mirrors": ["http://hub-mirror.c.163.com"],
  "insecure-registries":["192.168.1.102:8080"] 
}
# sudo systemctl restart docker

2. 安装软件

1
2
3
4
5
6
sudo wget -O /etc/yum.repos.d/jenkins.repo \
    https://pkg.jenkins.io/redhat-stable/jenkins.repo
sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
sudo yum upgrade
sudo yum install jenkins java-1.8.0-openjdk-devel
sudo systemctl daemon-reload
1
2
3
4
5
6
7
8
[root@jenkins ~]# wget -o /etc/yum.repos.d/jenkins.repo [https://pkg.jenkins.io/redhat-stable/jenkins.repo](https://pkg.jenkins.io/redhat-stable/jenkins.repo)
[root@jenkins ~]# rpm --import [https://pkg.jenkins.io/redhat-stable/jenkins.io.key](https://pkg.jenkins.io/redhat-stable/jenkins.io.key)
yum install -y jenkins git maven
systemctl start jenkins
[root@jenkins yum.repos.d]# lsof -i:8080
COMMAND    PID    USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    277770 jenkins   26u  IPv6 754801      0t0  TCP *:webcache (LISTEN)
ps -ef | grep jenkins

在CentOS 8 上安装Jenkins需要使用root账号或具有sudo权限的账号。

第一步、因为Jenkins是使用Java编写的，所以需要安装Java环境，Jenkins需要Java8或更高版本

这里使用OpenJDK，首先通过一下命令安装OpenJDK.

1
sudo dnf install java-1.8.0-openjdk-devel

如果系统上安装了多个Java版本，请确保Java 8是默认Java版本。

你可能对关于更详细如何在CentOS上安装Java教程也感兴趣。

第二步、启用Jenkins存储库。运行以下命令下载并导入GPG密钥：

1
2
sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key

第三步、通过键入以下命令安装最新的Jenkins稳定版本：

1
sudo dnf install jenkins

安装过程完成后，启动Jenkins服务并启用它以在系统引导时启动：

1
2
sudo systemctl start jenkins
sudo systemctl enable jenkins

检查Jenkins是否正在运行，请输入：

1
systemctl status jenkins

输出应类似如下所示：

1
2
3
Loaded: loaded (/etc/rc.d/init.d/jenkins; generated)
Active: active (running) since Thu 2019-11-05 21:31:36 UTC; 3s ago
...

2.1. 配置防火墙

如果要在受防火墙保护的远程CentOS服务器上安装Jenkins，则需要开放8080端口。

使用以下命令打开必要的端口：

1
2
sudo firewall-cmd --permanent --zone=public --add-port=8080/tcp
sudo firewall-cmd --reload

2.2. 设置Jenkins

安装完成并在防火墙上放开8080端口口，就可以开始Jenkins设置过程。首先，请打开浏览器并键入域或服务器的IP地址，然后键入port 8080：

1
http://your_ip_or_domain:8080

3.预先配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
由于在Jenkins机器上docker是使用root用户运行的，而Jenkins是使用普通用户jenkins运行的，所以要先配置下jenkins用户可以使用docker命令。
[root@jenkins ~]# visudo
jenkins ALL=(root)      NOPASSWD: /usr/bin/docker

另外在Jenkins机器上配置：

# Disable "ssh hostname sudo <cmd>", because it will show the password in clear.
#         You have to run "ssh -t hostname sudo <cmd>".
#
#Defaults    requiretty
Defaults:jenkins !requiretty

如果不配置这个，在执行下面脚本时，会报错误：
+ cp -f /home/jenkins/.jenkins/workspace/godseyeBranchForNov/godseye-container/target/godseye-container-wisedu.war /home/jenkins/docker-file/godseye_war/godseye.war
+ sudo docker login -u jkzhao -p Wisedu123 -e 01115004@wisedu.com 172.16.206.32
sudo: sorry, you must have a tty to run sudo

  1. 安装git遇到问题

    1
    2
    3
    4
    5
    6
    # 安装gcc环境
    # yum源没有对应版本包匹配

    wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-8.repo
    sed -i  's/$releasever/8/g' /etc/yum.repos.d/CentOS-Base.repo
    yum repolist

    image-20210822003946053

    image-20210822004504559

    jenkins配置:

  • 安装插件: maven intergation 和 ssh agent

  • 配置远程机器

    • 添加凭据

      • image-20210822012927566

        image-20210822013010640

        image-20210822013323484

        image-20210822145637999

六. jenkins构建job

1. 构建maven风格的job

1. 代码地址:

​ https://github.com/yzhou15/maven-docker

image-20210822151048548

image-20210822151119595

  • Goals and options填写：clean package -Dmaven.test.skip=true

image-20210822151142800

image-20210822151306263

image-20210822154617973

image-20210822155601445

2. Post steps

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
# jenkins机器: 编译完成后,build生成一个新版本的镜像,push到远程docker仓库

# Variables
JENKINS_WAR_HOME='/var/lib/jenkins/workspace/maven-docker/target'
DOCKERFILE_HOME='/var/lib/jenkins/docker-file/maven-docker-test_war'
HARBOR_IP='192.168.1.102'
REPOSTITORIES='jenkins/maven-docker'
HARBOR_USER='yzhou'
HARBOR_USER_PASSWS='Harbor12345'
HARBOR_USER_EMAIL='403708416@qq.com'

# Copy the nevest war to docker-file directory.
\cp -f ${JENKINS_WAR_HOME}/easy-springmvc-maven.war
${DOCKERFILE_HOME}/maven-docker.war

# Delete image early version
sudo docker login -u ${HARBOR_USER} -p ${HARBOR_USER_PASSWD} ${HARBOR_IP}:8080
IMAGE_ID='sudo docker images | grep ${REPOSITORIES} | awk '{print$3}''
if [ -n "{IMAGE_ID}"];then
	sudo docker rmi ${IMAGE_ID}
fi

# Build image.
cd ${DOCKERFILE_HOME}
TAG=`date +%Y%m%d-%H%M%S`
sudo docker build -t ${HARBOR_IP}:8080/${REPOSITORIES}:${TAG} . &>/dev/null

# Push to the harbor registry.
sudo docker push ${HARBOR_IP}:8080/${REPOSITORIES}:${TAG} &>/dev/null

3. 拉取镜像,发布

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# 拉取镜像.发布.
HARBOR_IP='192.168.1.102'
REPOSITORIES='jenkins/maven-docker'
HARBOR_USERS='yzhou'
HARBOR_USER_PASSWD='Harbor2345'

# 登录harbor
docker login -u ${HARBOR_USER} -p ${HARBOR_USER_PASSWD} ${HARBOR_IP}:8080

Stop container, and delete the container.
CONTAINER_ID=`docker ps | grep "maven-docker" | awk '{print$1}'`
if [ -n "$CONTAINER_ID" ]; then
	docker stop $CONTAINER_ID
	docker rm $CONTAINER_ID
else # 如果容器启动失败,就需要docker ps -a才能找到哪个容器
	CONTAINER_ID=`docker ps -a | grep "maven-docker" | awk 'print$1'`
		if [ -n "$CONTAINER_ID"]; then # 如果是第一次在这台及其上拉取运行容器,那么docker ps -a也是找不到这个容器的
			docker rm $CONTAINER_ID
		fi
fi

# Deleteeasy-springmvc-maven image early version.
IMAGE_ID=`sudo docker images | grep ${REPOSITORIES} | awk 'print $3'`
if [ -n "${IMAGE_ID}"];then
	docker rmi ${IMAGE_ID}
fi

# Pull images.
# TAG=`curl -s 
http://${HARBOR_IP}/api/repositories/${REPOSITORIES}/tags | jq '.[-1]' 
| sed 's/\"//g'`
TAG=`curl http://192.168.1.102/api/repoditories/jenkins/maven-docker/tags | jq '.[-1]' | {name:.name} | awk -F '"' '/name/{print $4}'`
docker pull ${HARBOR_IP}:8080/${REPOSITORIES}:${TAG} &>/dev/null}

# Run.
docker run -d --name maven-docker -p 8080:8080
${HARBOR_ip}:8080/${REPOSITORIES}:${TAG}

4. 项目构建

![Untitled 2](Git+Jenkins+Harbor+Docker实现CICD一些记录/Untitled 2-162964417842316.png)

5. 遇到问题:

image-20210822230454622

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# 查看maven版本
sudo dnf install maven
mvn -version
# 更新maven版本,Maven 3.3+需要安装JDK 1.7 或者更高版本的 JDK。
sudo dnf install java-11-openjdk-devel
java -version
# 下载 Apache Maven http://maven.apache.org/download.cgi
wget https://ftp.jaist.ac.jp/pub/apache/maven/maven-3/3.8.1/binaries/apache-maven-3.8.1-bin.tar.gz -P /tmp
sudo tar xf /tmp/apache-maven-3.8.1-bin.tar.gz -C /opt
sudo ln -s /opt/apache-maven-3.8.1 /opt/maven
# 设置环境变量
# vim /etc/profile.d/maven.sh
export JAVA_HOME=/usr/lib/jvm/jre-openjdk
export M2_HOME=/opt/maven
export MAVEN_HOME=/opt/maven
export PATH=${M2_HOME}/bin:${PATH}

sudo chmod +x /etc/profile.d/maven.sh
source /etc/profile.d/maven.sh
# 验证安装
[root@jenkins ~]# mvn -version
Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d)
Maven home: /opt/maven
Java version: 1.8.0_302, vendor: Red Hat, Inc., runtime: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.302.b08-0.el8_4.x86_64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.18.0-305.10.2.el8_4.x86_64", arch: "amd64", family: "unix"

6. 控制台输出过程

image-20210822230419964

1
# 修改git分支为默认

image-20210822164823295

image-20210822230534472

1
2
3
4
# jenkins找不到,新增maven
mvn -v
# 配置全路径,清理工作空间
$workspace/pom.xml