How to Setup Jenkins Master-Slave Architecture with Ubuntu 24.04

An efficient Continuous Integration and Continuous Delivery (CI/CD) pipelines are crucial. As your projects grow and your team expands, your Jenkins server might start feeling the strain. This is where the Jenkins Master-Slave Architecture comes to the rescue!

This guide will equip you with a step-by-step understanding of how to set up a Jenkins Master-Slave architecture, ensuring your CI/CD pipelines remain lightning-fast and reliable, even under heavy workloads.

By the end of this comprehensive guide, you'll be able to answer:

• ✅ What is Jenkins Master-Slave Architecture (and why is it now called Master-Agent)?
• ✅ Why is this architecture essential for scaling your CI/CD?
• ✅ How to set up a robust Jenkins Master-Agent setup on Ubuntu 24.04 LTS (step-by-step)?
• ✅ Explore practical example configurations and sample pipeline scripts.

Let's get started and unlock the true potential of your Jenkins setup!

What is Jenkins Master-Agent Architecture?

The Jenkins Master-Agent architecture is a distributed system designed to enhance the performance and scalability of your Jenkins environment. It involves two core components:

• Jenkins Master (Controller): The central brain of your Jenkins setup. The Master is responsible for:

  • Scheduling build jobs.
  • Dispatching jobs to Agents (Slaves).
  • Monitoring Agents.
  • Presenting the web UI and overall Jenkins management.
  • Storing build results and configurations.

• Jenkins Agents (Slaves): Worker nodes that execute the actual build jobs dispatched by the Master. Agents are designed to be lightweight and can run on various operating systems and environments. They connect to the Master and wait for job assignments.

Why the Name Change? You might hear "Master-Agent" more frequently than "Master-Slave." This is a move towards more inclusive and respectful language within the tech community, and reflects the functional relationship accurately. While the underlying architecture remains the same, "Agent" is the preferred and modern term for the worker nodes.

Why Scale with Jenkins Agents? The Need for Distributed Builds

As your projects and teams grow, a single Jenkins Master can become a bottleneck. Here's why a Master-Agent architecture becomes crucial:

• Improved Performance: Distribute build workloads across multiple Agents, significantly reducing build times and improving overall pipeline speed.
• Enhanced Scalability: Easily scale your CI/CD infrastructure by adding more Agents as your project demands increase. No more waiting in build queues!
• Increased Stability and Resilience: Isolate build execution to Agents. If an Agent fails, the Master remains operational, and jobs can be rescheduled on other available Agents.
• Support for Diverse Environments: Agents can run on different operating systems (Linux, Windows, macOS) and architectures, allowing you to build and test across various platforms within a single Jenkins setup.
• Resource Optimization: Offload resource-intensive build tasks from the Master, freeing up its resources for management and scheduling.

Essentially, Jenkins Agents empower you to handle complex CI/CD pipelines efficiently, ensuring faster feedback loops and quicker software delivery.

Step-by-Step Guide

Let's walk through the process of setting up a Jenkins Master and Agent on Ubuntu 24.04 LTS (Noble Numbat).

Prerequisites:

• Two or more Ubuntu 24.04 LTS servers (or VMs): One for the Jenkins Master and at least one for a Jenkins Agent.
• Java Runtime Environment (JRE) or Java Development Kit (JDK) 8 or 11: Ensure Java is installed on both Master and Agent nodes.
• SSH Access: SSH access configured between the Master and Agent nodes for seamless communication.
• Network Connectivity: Master and Agent nodes should be able to communicate with each other over the network.

Step 1: Install Jenkins Master on Ubuntu 24.04 LTS

1. Update Package Index:

   Bash

   sudo apt update
   

2. Install Java (if not already installed):

   Bash

   sudo apt install openjdk-11-jre
   

3. Add Jenkins Repository:

   Bash

   curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo gpg --dearmor -o /usr/share/keyrings/jenkins.gpg
   echo deb [signed-by=/usr/share/keyrings/jenkins.gpg] https://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null
   

4. Update Package Index Again:

   Bash

   sudo apt update
   

5. Install Jenkins:

   Bash

   sudo apt install jenkins
   

6. Start and Enable Jenkins Service:

   Bash

   sudo systemctl start jenkins
   sudo systemctl enable jenkins
   sudo systemctl status jenkins # Verify Jenkins is running
   

7. Access Jenkins Web UI: Open your web browser and navigate to http://<your_master_server_ip>:8080. Follow the initial setup wizard to configure your Jenkins Master.

Step 2: Create a Dedicated Jenkins User on the Agent Node (Ubuntu 24.04 LTS)

For security best practices, create a dedicated user for Jenkins Agent on the Agent server:

1. Create the jenkins-agent user:

   Bash

   sudo adduser jenkins-agent
   

   Set a password for this user when prompted.

2. Add jenkins-agent user to docker group (if using Docker on Agent):

   Bash

   sudo usermod -aG docker jenkins-agent
   

Step 3: Install Java and Agent Dependencies on the Agent Node (Ubuntu 24.04 LTS)

1. Install Java (if not already installed):

   Bash

   sudo apt update
   sudo apt install openjdk-11-jre
   

2. Install any other dependencies required for your build jobs: This might include tools like git, maven, gradle, docker, etc., depending on your project's needs.

Step 4: Connect the Agent Node to the Jenkins Master

1. Log in to your Jenkins Master Web UI.

2. Navigate to "Manage Jenkins" > "Manage Nodes and Clouds" > "New Node".

3. Enter a Node Name (e.g., agent-node-01). Select "Permanent Agent" and click "OK".

4. Configure the Agent Node:

   • Name: agent-node-01 (or your chosen name)
   • Description: (Optional) Description of the Agent.
   • Remote root directory: /home/jenkins-agent/agent (This directory will be created on the Agent node)
   • Labels: ubuntu-agent (or any label to identify this agent)
   • Launch method: "Launch agents via SSH"
   • Host: IP address or hostname of your Agent server.
   • Credentials: Add new credentials. Choose "Username with password".
     • Username: jenkins-agent
     • Password: Password you set for the jenkins-agent user.
     • ID: jenkins-agent-credentials (or any ID for your credentials)
   • Host Key Verification Strategy: "Manually trusted key verification" (or "Known hosts file strategy" if you prefer)
   • JavaPath: (Leave empty to use the default java on the Agent's PATH)

5. Click "Save".

6. Check Agent Connection: Jenkins will attempt to connect to the Agent. You should see the Agent status change from "Offline" to "Idle" (or "Online") in the "Manage Nodes and Clouds" page.

Step 5: Configure Jobs to Run on Agents

Now that your Agent is connected, you can configure your Jenkins jobs to run on it:

1. Open your Jenkins Job configuration.

2. Navigate to the "Restrict where this project can be run" section.

3. Enter the Label you assigned to your Agent (e.g., ubuntu-agent). This will ensure that this job will only be executed on Agents with the ubuntu-agent label.

4. Save the Job configuration.

Example Configurations & Sample Pipeline Script

Let's look at a simple example pipeline script that utilizes our newly configured Agent:

Groovy

pipeline {
    agent {
        label 'ubuntu-agent' // Run this pipeline on any agent with the 'ubuntu-agent' label
    }
    stages {
        stage('Checkout') {
            steps {
                git 'https://github.com/your-repo/your-project.git'
            }
        }
        stage('Build') {
            steps {
                sh 'mvn clean install' // Example Maven build command
            }
        }
        stage('Test') {
            steps {
                sh 'mvn test' // Example Maven test command
            }
        }
        stage('Deploy') {
            steps {
                sh './deploy.sh' // Example deployment script
            }
        }
    }
}

Explanation:

• agent { label 'ubuntu-agent' }: This line in the pipeline script tells Jenkins to execute this pipeline on any Agent node that has the label ubuntu-agent.
• The subsequent stages (Checkout, Build, Test, Deploy) will then be executed on the designated Agent, offloading the workload from the Jenkins Master.

Conclusion

Congratulations! You've successfully set up a Jenkins Master-Agent architecture on Ubuntu 24.04 LTS. By distributing your build workloads, you've significantly enhanced the performance, scalability, and resilience of your CI/CD pipelines.