Jenkins Master Slave Setup

Due to increasing demand of Continuous integration and Continuous delivery (CI/CD), most of us has heard the name of three most popular tools that are

• Jenkins
• Git Lab
• GitHub Actions

In this article, I’m going to cover Jenkins with its setup and master slave configuration.

Introduction

Jenkins is a self-contained, open source automation server which can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software. In layman term, Jenkins run jobs or tasks when mentioned condition meet. Example, as soon as there is a push on your GitHub repository it should clone it on the server, test and deploy the changes.

Installation

I’m using Ubuntu 22.04 as my operating system but Jenkins is available for different platform and the installation process is mentioned here.

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins

After installation check the status of Jenkins using

sudo systemctl status jenkins # it should be up and running, else
sudo systemctl start jenkins # to start Jenkins

jenkins --version # mine is 2.414.2

By default it runs on port 8080. As soon as you land on this port with your machine’s IP, the Jenkins homepage would show up and ask you to follow some steps (changing password, installing plugins) with mentioned instruction.

Note: While asking for plugin installation choose recommended one not the custom one if you are just starting with Jenkins.

Jenkins Homepage (image 1)

After Jenkins has been configured, jobs can be run directly on the node where it is installed, but it is generally not a good idea to run jobs directly on the master node.

Jenkins Nodes (Image 2)

Master Slave Configuration

Let’s add a worker node(machine/server) where our jobs will run and leave master as it is.

• The Jenkins on master can connect to worker node using i) SSH with username and password ii) SSH with keys.
• We are going to use SSH with keys.

Steps

• Launch a machine(VM/EC2).
• Create SSH keys

# My username is worker1
ssh-keygen -t ed25519 # with rsa type it was not working hence I had to use ed25519

cd .ssh && touch authorized_keys
cat id_ed25519.pub >> authorized_keys

# create a directory where jenmins can create its workspace
# and store info about jobs

sudo mkdir /opt/jenkins
sudo chown worker1:worker1 -R /opt/jenkins

• Go to Jenkins UI
• Dashboard → Manage Jenkins → Nodes

Image 3

• Open Nodes → New Node (Image 2)
• Enter Name of node → click on Permanent Agent

Setup Node’s name (Image 4.1)

• Click on Create
• There would be bunch of text field in which we would change some.

Setup Worker Node (Image 4.2)

• Name → Unique name to identify the agent.
• Description → mention what kind of jobs is going to run here
• Number of Executors → Number of jobs that can run in parallel(concurrent jobs)
• Remote Directory → Directory on worker node where Jenkins will create the workspace and run the jobs.
• Labels → This one is quite important as jobs identify the nodes via labels. We will see a example on this.
• Usage → There are two options here i) Use this node as much as possible ii) Only build jobs with label expression matching this node. The first one allow the jobs to be executed on this node in which wither no worker nodes are allowed or label matching to this node’s label whereas the second option allow only those jobs in which the label are mentioned as “first-worker”.

Setup Worker Node (Image 4.3)

• Launch Method → Here “Launch agents via SSH” needs to be selected as we are going to use SSH with keys.
• Host → IP or DNS of the machine going to be the worker node.
• Credentials → Currently in your case, you don’t have any credentials for authentication, so to setup that click on ADD then select Jenkins. Please check the image 4.4 below this

Setup Worker Node (Image 4.4)

• Kind → It should be SSH username with Private key
• Domain and Scope can be global as in image.
• ID → Unique identifier for cred.
• username → username used while SSH
• private key → Earlier we generated ssh keys and by default the name of the file is “id_ed25519 and id_ed25519.pub” . id_ed25519 is a private key, copy the value and paste it under private key section (Image 4.5). There should be no new line after the key.

Setup Worker Node (Image 4.5)

• Click on Add then select the Credential(Image 4.3) from drop down.
• Match the other option from the image.

Click on SAVE.

Worker Node successfully connected (Image 5)

• If it doesn’t get connected, click on the node’s name then check the logs from left panel.
• Check the failure cause. The two main reason are i) authentication failed ii) Jenkins is not able to write into root working directory (/opt/jenkins)
• For authentication one, you can try to make that node offline for a minute then enable it again. It should resolve then else check the permission of keys.
• For directory, change the owner and group.

Run the Job on Worker Node

Create Job (Image 6)

• click on new item
• Enter item name that is job name and select Freestyle project
• After that while configuring click on “Restrict where this project can be run” and enter the worker node’s label.

Enter label’s name(Image 7)

Shell execution(Image 8)

Job Output (Image 9)

• Here, the job is running on mentioned node.

Summary

Jenkins is a CI/CD tool that offers master slave setup so that jobs can be distributed according to the resource needed and doesn’t take down the master itself.

Try creating more nodes and run multiple jobs at a time.

Thanks for reading!!!