GoCD is an open source continuous delivery and automation system. It allows you to model complex workflows using its parallel and sequential execution. Its value stream map allows you to easily visualize a complex workflow with ease. GoCD lets you easily compare two builds and deploy any version of the application you want. The GoCD ecosystem consists of GoCD server and GoCD agent. GoCD is responsible for controlling everything such as running the web-based user interface and managing and providing jobs to the agent. Go agents are responsible for running the jobs and deployments.
A Vultr CentOS 7 server instance with at least 1GB RAM.
A sudo user.
A domain name pointed towards the server.
For this tutorial, we will use 192.168.1.1 as the public IP address and gocd.example.com as the domain name pointed towards the Vultr instance. Please make sure to replace all occurrences of the example domain name and IP address with the actual one.
Update your base system using the guide How to Update CentOS 7. Once your system has been updated, proceed to install Java.
GoCD requires Java version 8 and supports both Oracle Java and OpenJDK. In this tutorial, we will be installing Java 8 from OpenJDK.
OpenJDK can be easily installed, as the package is available in the default YUM repository.
sudo yum -y install java-1.8.0-openjdk-devel
If Java is installed correctly, then you will be able to verify its version.
java -version
You will get a similar output to the following text.
[user@vultr ~]$ java -version
openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)
Before we can proceed further, we will need to set up the JAVA_HOME and JRE_HOME environment variables. Find the absolute path of the Java executable in your system.
readlink -f $(which java)
The following text will be outputted to your terminal.
[user@vultr ~]$ readlink -f $(which java)
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.151-5.b12.el7_4.x86_64/jre/bin/java
Now, set the JAVA_HOME and JRE_HOME environment variables according to the path of the Java directory.
echo "export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.151-5.b12.el7_4.x86_64" >> ~/.bash_profile
echo "export JRE_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.151-5.b12.el7_4.x86_64/jre" >> ~/.bash_profile
Note: Make sure you use the Java path obtained on your system. The path used in this tutorial may change when a new version of Java 8 is released.
Execute the bash_profile file.
source ~/.bash_profile
Now you can run the echo $JAVA_HOME command to ensure that the environment variable is set.
[user@vultr ~]$ echo $JAVA_HOME
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.151-1.b12.el7_4.x86_64
GoCD is written in Java, hence Java is the sole dependency to run GoCD. GoCD can be installed with the help of YUM. Install its official repository into the system.
sudo curl https://download.gocd.org/gocd.repo -o /etc/yum.repos.d/gocd.repo
Install the GoCD server in your system.
sudo yum install -y go-server
Start GoCD and enable it to automatically start at boot time.
sudo systemctl start go-server
sudo systemctl enable go-server
Before we access the GoCD dashboard, let's create a new directory to store the artifacts. Artifacts can be stored on the same disk in which the operating system and the applications are installed. Alternatively, you can use a dedicated disk or block storage drive to store the artifacts.
If you wish to use the same disk to store the artifacts, just create a new directory and provide the ownership to the GoCD user.
sudo mkdir /opt/artifacts
sudo chown -R go:go /opt/artifacts
The GoCD software recommends to use an additional partition or drive to store the artifacts. In a continuous integration and delivery platform, artifacts are generated very often. The disk space decreases over time when new artifacts are continuously generated. At some stage, your system will run out of free disk space and the services running on your system will fail. To overcome this issue, you can attach a new Vultr block storage drive to store the artifacts. If you still wish to go with the storage of artifacts on the same drive, skip to the "Setup Firewall" section.
Deploy a new block storage drive and attach it to your GoCD server instance. Now create a new partition on the block storage device.
sudo parted -s /dev/vdb mklabel gpt
sudo parted -s /dev/vdb unit mib mkpart primary 0% 100%
Create the file system on the new disk.
sudo mkfs.ext4 /dev/vdb1
Mount the block storage drive.
sudo mkdir /mnt/artifacts
sudo cp /etc/fstab /etc/fstab.backup
echo "
/dev/vdb1 /mnt/artifacts ext4 defaults,noatime 0 0" | sudo tee -a /etc/fstab
sudo mount /mnt/artifacts
Now, run df, and you will see the new block storage drive mounted on /mnt/artifacts.
[user@vultr ~]$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/vda1 20616252 6313892 13237464 33% /
...
/dev/vdb1 10188052 36888 9610596 1% /mnt/artifacts
Provide the ownership of the directory to the GoCD user.
sudo chown -R go:go /mnt/artifacts
Change the firewall configuration to allow ports 8153 and 8154 through the firewall. Port 8153 listens for unsecured connections and port 8154 for secured connections.
sudo firewall-cmd --zone=public --add-port=8153/tcp --permanent
sudo firewall-cmd --zone=public --add-port=8154/tcp --permanent
sudo firewall-cmd --reload
Now you can access the GoCD dashboard on http://192.168.1.1:8153. To access the GoCD dashboard on a secured connection, access https://192.168.1.1:8154. You will get some error showing that the certificates are not valid. You can safely ignore the error as the certificates are self-signed. For security purposes, you should always use the dashboard over a secured connection.
Before you set up a new pipeline, navigate to "Admin >> Server Configuration" from the top navigation bar.
Input the URL to your unsecured site in the "Site URL" field and secured site in the "Secure Site URL" field.
Next, provide your SMTP server details to send email notifications from GoCD.
Finally, provide the path to the location where you wish to store the artifacts. If you have chosen to store the artifacts on the same disk as the operating system, enter /opt/artifacts; if you have chosen to attach a block storage drive, then you can enter /mnt/artifacts.
Also, you can configure GoCD to auto-delete the old artifacts. Configure the next option according to your disk size. However, the auto-delete option does not take a backup of your old artifacts. To manually take a backup and then delete the old artifacts, disable auto delete by choosing the "Never" option for the "Auto delete old artifacts" option.
You will need to restart the GoCD server so that the new changes are applied.
sudo systemctl restart go-server
By default, the GoCD dashboard is not configured to use any kind of authentication, but it supports authentication using a password file and LDAP. In this tutorial, we will set up password-based authentication.
Note: Setting up authentication is an optional step, but it is strongly recommended for public facing servers, such as Vultr.
Install Apache tools so that we can use the htpasswd command to create an encrypted password file.
sudo yum -y install httpd-tools
Create a password file with the htpasswd command using Bcrypt encryption.
sudo htpasswd -B -c /etc/go/passwd_auth goadmin
Provide the password for the user twice. You will see the following output.
[user@vultr ~]$ sudo htpasswd -B -c /etc/go/passwd_auth goadmin
New password:
Re-type new password:
Adding password for user goadmin
You can add as many users as you want using the same command above, but removing the -c option. The -c option will replace the existing file, replacing old users with the new user.
sudo htpasswd -B /etc/go/passwd_auth gouser1
Since, we have created the password file, access the GoCD dashboard again. Navigate to "Admin >> Security >> Authorization Configurations" from the top navigation bar. Click on the Add button and provide an ID. Choose "Password File Authentication Plugin for GoCD" for plugin ID and direct the path to the password file. Now click on the "Check Connection" button to verify that GoCD can use the password file for authentication.
Finally, save the authentication method. Reload the dashboard and it will automatically log you out. You will see a login screen now. Log in using the credentials created earlier.
You will need to promote the administrator user manually, otherwise, all the users will have administrator privileges. Navigate to "Admin >> User Summary" from the top navigation bar.
Now select the admin user you've created and click on the "Roles" drop-down. Promote the user to the only administrator by selecting the "Go System Administrator" checkbox.
To add the users in GoCD created in the password file, click on the "ADD" button and search for the user to add them. Users are also automatically added to the GoCD dashboard on their first login. Obviously, for users to log in, they must be added to the password file we have created earlier.
By default, GoCD listens to ports 8153 and 8154 on secure connections. Though port 8154 provides a secure connection to the application, it also displays browser errors as it uses a self-signed certificate. In this section of the tutorial, we will install and secure Nginx with Let's Encrypt free SSL certificate. The Nginx web server will work as a reverse proxy to forward the incoming requests to GoCD's HTTP endpoint.
Install Nginx.
sudo yum -y install nginx
Start Nginx and enable it to automatically start at boot time.
sudo systemctl start nginx
sudo systemctl enable nginx
Install Certbot, which is the client application for Let's Encrypt CA.
sudo yum -y install certbot
Before you can request certificates, you will need to allow ports 80 and 443, or standard HTTP and HTTPS services, through the firewall. Also, remove port 8153, which listens to the unsecured connections.
sudo firewall-cmd --zone=public --add-service=http --permanent
sudo firewall-cmd --zone=public --add-service=https --permanent
sudo firewall-cmd --zone=public --remove-port=8153/tcp --permanent
sudo firewall-cmd --reload
Note: To obtain certificates from Let's Encrypt CA, the domain for which the certificates are to be generated must be pointed towards the server. If not, make the necessary changes to the DNS records of the domain and wait for the DNS to propagate before making the certificate request again. Certbot checks the domain authority before providing the certificates.
Generate the SSL certificates.
sudo certbot certonly --webroot -w /usr/share/nginx/html -d gocd.example.com
The generated certificates are likely to be stored in /etc/letsencrypt/live/gocd.example.com/. The SSL certificate will be stored as fullchain.pem and private key will be stored as privkey.pem.
Let's Encrypt certificates expire in 90 days, so it is recommended to set up auto-renewal of the certificates using cron jobs.
Open the cron job file.
sudo crontab -e
Add the following line to the end of the file.
30 5 * * * /usr/bin/certbot renew --quiet
The above cron job will run every day at 5:30 AM. If the certificate is due for expiry, it will automatically renew.
Now, change the Nginx default configuration file to take out the default_server line.
sudo sed -i 's/default_server//g' /etc/nginx/nginx.conf
Create a new configuration file for the GoCD web interface.
sudo nano /etc/nginx/conf.d/gocd.conf
Populate the file.
upstream gocd {
server 127.0.0.1:8153;
}
server {
listen 80 default_server;
server_name gocd.example.com;
return 301 https://$host$request_uri;
}
server {
listen 443 default_server;
server_name gocd.example.com;
ssl_certificate /etc/letsencrypt/live/gocd.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/gocd.example.com/privkey.pem;
ssl on;
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/gocd.access.log;
location / {
proxy_pass http://gocd;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_redirect off;
}
location /go {
proxy_pass http://gocd/go;
proxy_http_version 1.1;
proxy_set_header Upgrade websocket;
proxy_set_header Connection upgrade;
proxy_read_timeout 86400;
}
}
Check for errors in the new configuration file.
sudo nginx -t
If you see the following output, the configuration is error free.
[user@vultr ~]$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
If you have received some kind of error, make sure to double check the path to the SSL certificates. Restart the Nginx web server to implement the change in configuration.
sudo systemctl restart nginx
Now you can access the GoCD dashboard at https://gocd.example.com. Log in to your dashboard using the administrator credentials and navigate to "Admin >> Server Configuration" from the top navigation bar.
Set the "Site URL" and "Secure Site URL" to https://gocd.example.com. Port 8154 still needs to be accessible through the firewall so that the remote agents may connect to the server through port 8154, in case they are unable to connect through the standard HTTP port.
In the GoCD continuous integration environment, GoCD agents are the workers that are responsible for the execution of all the tasks. When a change in the source is detected, the pipeline is triggered and the jobs are assigned to the available workers for execution. The agent then executes the task and reports the final status after execution.
To run a pipeline, at least one agent must to be configured. Proceed to install the GoCD agent on the GoCD server.
Since we have already imported the GoCD repository into the server, we can directly install Go Agent.
sudo yum install -y go-agent
Now, start the GoCD server and enable it to automatically start at boot time.
sudo systemctl start go-agent
sudo systemctl enable go-agent
The GoCD agent running on the localhost is automatically enabled when detected.