Skip to main content

Jenkins

Overview​

The main difference between Jenkins and Semaphore is that Semaphore is a managed service while Jenkins is purely self-hosted.

In Jenkins you are in charge of configuring everything, installing plugins for all the functionality you need, managing the agents to run the workflows, creating the connections to the Git providers, managing the Jenkins instance, and the list goes on.

Semaphore is always ready to use, once you create an account and connect your Git provider you're ready to go. There is nothing to manage and you get first-class support.

Jenkins vs Semaphore​

This section describes how to implement common Jenkins functionalities in Semaphore.

Checkout​

Checkout clones the repository in the CI system. This is usually near the beginning of every job and workflow.

In Jenkins, we use the Git plugin to connect and retrieve the repository history. You need to add authentication credentials on the Jenkins instance and use them in the stage.

stage('Checkout repository') {
    steps {
        git branch: 'main',
            credentialsId: '<my-repo-auth>',
            url: 'git@github.com:<owner>/<project-name>.git'

        sh "cat README.md"
    }
}

Artifacts​

Artifacts are used to store deliverables and persist files between runs.

In Jenkins we use archiveArtifacts to store files:

stage('Build') {
    steps {
        // Your build steps here
        // ...
        
        archiveArtifacts artifacts: 'build/output/**/*.jar', fingerprint: true
    }
}

And copyArtifacts to retrieve them:

stage('Deploy') {
    steps {
        copyArtifacts(projectName: 'MyProject', filter: '**/*.jar', target: 'deploy-directory')
    }
}

Caching​

The cache speeds up workflows by keeping a copy of dependencies in storage.

In Jenkins, we need to install the jobcacher plugin to enable the cache. Then, we a cache stage to the workflow before building the project.

stage('Cache Dependencies') {
     steps {
         cache(maxCacheSize: 250, caches: [
             [$class: 'ArbitraryFileCache', 
              includes: ['**/node_modules/**'], 
              excludes: [],
              path: 'node_modules',
              fingerprintingStrategy: [$class: 'DefaultFingerprintStrategy']]
         ]) {
             // Your build steps go here
            sh 'npm install'
        }
    }
}

Language versions​

We often need to activate specific language or tool versions to ensure consistent builds.

Jenkins doesn't have a native way to activate languages. That means you have to install a plugin or run the language installation commands manually in a stage.

stage('Setup Go') {
    steps {
        sh '''
            source ~/.gvm/scripts/gvm
            gvm install go1.21.0  # or whatever version you need
            gvm use go1.21.0
            go version
        '''
    }
}

stage('Build') {
    steps {
        sh 'go build'
    }
}

Databases and services​

Testing sometimes requires disposable databases and services in the CI environment.

Jenkins has plugins for various databases and services. It also supports running services with Docker, which is often the easiest way to run disposable instances.

stages {
    stage('Start Database') {
        steps {
            sh 'docker run --name test-postgres -e POSTGRES_PASSWORD=mysecretpassword -d postgres'
            // Wait for database to be ready
            sh 'sleep 10'
        }
    }
    
    stage('Run Tests') {
        steps {
            sh 'npm test'
        }
    }
    
    stage('Cleanup') {
        steps {
            sh 'docker stop test-postgres'
            sh 'docker rm test-postgres'
        }
    }
}

Secrets​

Secrets inject sensitive data and credentials into the workflow securely.

In Jenkins, we create the credentials at the instance level and then initialize variables using the credentials id.

environment {
        AWS_ACCESS_KEY_ID     = credentials('jenkins-aws-secret-key-id')
        AWS_SECRET_ACCESS_KEY = credentials('jenkins-aws-secret-access-key')
}

// later in stages ...

    stage('AWS S3 Access') {
        steps {
            sh 'aws s3 ls'
        }
    }

Complete example​

The following comparison shows how to build and test a Ruby on Rails project on Jenkins and on Semaphore.

This pipeline runs all the tests in different sequential stages.

pipeline {
    agent any

    environment {
        RAILS_ENV = 'test'
    }

    stages {
        stage('Scan Ruby') {
            agent { label 'ubuntu' }
            steps {
                // Checkout code
                checkout scm

                // Set up Ruby
                sh '''
                    curl -sSL https://get.rvm.io | bash -s stable --ruby=$(cat .ruby-version)
                    source /usr/local/rvm/scripts/rvm
                    bundle install
                '''

                // Scan for common Rails security vulnerabilities
                sh 'bin/brakeman --no-pager'
            }
        }

        stage('Scan JS') {
            agent { label 'ubuntu' }
            steps {
                // Checkout code
                checkout scm

                // Set up Ruby
                sh '''
                    curl -sSL https://get.rvm.io | bash -s stable --ruby=$(cat .ruby-version)
                    source /usr/local/rvm/scripts/rvm
                    bundle install
                '''

                // Scan for security vulnerabilities in JavaScript dependencies
                sh 'bin/importmap audit'
            }
        }

        stage('Lint') {
            agent { label 'ubuntu' }
            steps {
                // Checkout code
                checkout scm

                // Set up Ruby
                sh '''
                    curl -sSL https://get.rvm.io | bash -s stable --ruby=$(cat .ruby-version)
                    source /usr/local/rvm/scripts/rvm
                    bundle install
                '''

                // Lint code for consistent style
                sh 'bin/rubocop -f github'
            }
        }

        stage('Test') {
            agent { label 'ubuntu' }
            steps {
                // Install packages
                sh 'sudo apt-get update && sudo apt-get install --no-install-recommends -y curl libjemalloc2 libvips sqlite3'

                // Checkout code
                checkout scm

                // Set up Ruby
                sh '''
                    curl -sSL https://get.rvm.io | bash -s stable --ruby=$(cat .ruby-version)
                    source /usr/local/rvm/scripts/rvm
                    bundle install
                '''

                // Run Rake tasks
                sh '''
                    cp .sample.env .env
                    bundle exec rake db:setup
                    bundle exec rake
                '''

                // Run tests
                sh 'bin/rails db:test:prepare test test:system'
            }
        }
    }
    post {
        always {
            // Clean up
            deleteDir()
        }
    }
}

See also​