Creating Your First CI/CD Pipeline#
This step-by-step tutorial will have you running a successful CI/CD pipeline on Semaphore in minutes. Let's get started!
Semaphore is a cloud-based automation service for building, testing, and deploying software.
Semaphore is built for developer productivity and guided by three principles:
- Speed: Developers work in a fast feedback loop, so CI/CD workflows must be fast.
- Power: The CI/CD tool needs to be able to run any automated software workflow, at any scale.
- Ease of use: CI/CD must be easy enough for any developer to use so that they remain in close contact with the operation of their software and its impact on users.
To complete this tutorial, you will need:
- Basic knowledge of Git
- Basic knowledge of the command line
- A Semaphore account
- A GitHub or Bitbucket account
Creating a Git repository#
Start by creating a new Git repository on GitHub or Bitbucket. You may skip this section if you plan to use an existing repository.
- Create a new repository on Github or Bitbucket
- Enter the name of your repository, for example
hello-semaphore, then click Initialize this repository with: Add a README file. Finally, click Create repository.
Creating a Semaphore project#
- Log in to your Semaphore account.
- If you are part of multiple organizations, select the one in which you plan to host the project you are about to create.
- In the navigation panel, click Create new.
- In the Start a real project section, click Choose repository.
- Follow the on-screen instructions to authorize Semaphore to access a Git repository of your choice. In case you need help, consult the detailed guide to connecting your GitHub or Bitbucket account with Semaphore.
- Wait a few seconds for Semaphore to initialize the project.
- The next screen allows you to invite other people who have access to the repository to your Semaphore project. Click Continue to workflow setup to proceed.
- Use one of the templates to get to the first green build quickly: select Single job, then click Looks good, start.
- You will see your first workflow running and within seconds it should finish successfully.
Adding a test block to your CI pipeline#
Let's customize your pipeline so that it looks more like a continuous integration (CI) process for a real project.
If you open your repository on GitHub or Bitbucket, you will notice that there is a new
branch: setup-semaphore. Opening this branch will reveal that Semaphore has
semaphore.yml configuration file into the
Semaphore, like most modern CI/CD tools, creates workflows based on YAML configuration files. However, you don't need to write YAML by hand. Semaphore features a visual Workflow Builder, which lets anyone create and modify pipelines using a point-and-click interface. Of course, you can also always edit your configuration by hand in your favorite editor.
Adding a block with parallel jobs using Workflow Builder#
To extend your pipeline:
- On the workflow page, click Edit Workflow.
- Inside Initial pipeline, click Add block.
- In the pane on the right-hand side, change Name of the block to
- Change the name of the first job from
Unit tests, and in the Commands text area, enter
echo 'running unit tests'.
- Click on the Add another job link to add a parallel job. Name it
Integration testsand enter
echo 'running integration tests'into the commands text area.
- Click Prologue and enter
checkoutas a command to run. This is a built-in script that downloads the content of your Git repository. By placing it in the prologue, we reuse it as a starting command in both parallel jobs.
- In the top-right corner of your screen, click Run the workflow.
- In the pop-up screen, you can view the diff in the YAML
that Semaphore will append to
.semaphore/semaphore.yml. Click Looks good, start. This will push the changes to the Git repository and start a new workflow.
At this point, you have a multi-stage CI pipeline with parallel jobs:
Understanding the basic building blocks of Semaphore workflows#
Semaphore runs a workflow on each change in your Git repository. Your last
workflow, for example, contains a single pipeline called
However, workflows can be composed of more than one pipeline. Additional pipelines can be triggered manually or automatically based on custom conditions. We will see an example of that in a minute.
A pipeline can contain multiple blocks. For example, your current pipeline
contains two blocks:
Block #1 runs first and the
block runs when it completes.
Blocks can contain a single job or many parallel jobs. For example, your
Tests block contains two parallel jobs.
Running parallel jobs simultaneously is possible under open source, free trial, or paid plans.
Jobs are basic units of execution. Semaphore runs each job as a sequence of
commands in a clean, isolated environment that it creates on-demand and destroys
as soon as the last command is finished. If the exit code of each command is
0, Semaphore records the job as successful (passed).
In your workflow, you can click Unit tests to view the output log of this job. You
will see the command
echo 'running unit tests' that you wrote previously.
Clicking on it will expand its output.
You will also notice that Semaphore transparently displays all the commands that it performs to prepare your job for execution. This enables you to have full insight and control over your CI/CD environment.
Adding a CD pipeline#
Semaphore has first-class support for continuous delivery (CD) via promotions, which trigger other pipelines. By combining promotions and pipelines, you can:
- Manage multiple release targets
- Implement any deployment strategy
- Perform rollbacks
- Manage infrastructure
- Automate any DevOps task with a full log of who did what and when
Adding an automatic promotion and a deployment pipeline#
Let's extend our workflow by implementing continuous deployment to a
production environment when the CI pipeline on the
main branch has passed.
- From your workflow or job, click Edit Workflow.
- On the right-hand side following your
Initial Pipeline, click Add First Promotion.
Production deploymentas the name of your promotion.
- Check Enable automatic promotion.
- As conditions for automatic promotion, enter
branch = 'main' AND result ='passed'.
- If needed, scroll to the right, so that a new pipeline is visible on your screen. By default, it will be called
Pipeline 2. You can always change this later.
- Click on
Block #1and rename the block to
- Rename the job to
Deploy to production.
- Add a command
echo "deploying using key $API_KEY".`
- Click Environment variables, then click Add env vars. Define
- Click Run the workflow.
When this workflow runs, it will not run your promotion. This is because we are
still working in the
setup-semaphore branch, while we configured our
promotion to run automatically only on the
Running a promotion manually#
Before merging to main, we can verify our promotion by running it manually:
- On the workflow page, click Production deployment then Start promotion.
- Click on
Pipeline 2to expand the deployment pipeline. Et voilà!
To finish the implementation of your Semaphore project, open a pull request on
GitHub or Bitbucket from the
After you merge, Semaphore will trigger a new workflow on the
main branch, and
it will run both the CI and CD pipelines.
To practice what you just learned further, here are a few ideas that you could do as an exercise:
- Edit any of the Semaphore configuration files by hand. Does pushing the change trigger a new workflow? Is the change reflected in the Workflow Builder?
- Add one more block to the production deployment pipeline that would perform a smoke test.
- Add another promotion for manually deploying to a staging environment.
Where to go next#
In this tutorial, we covered the very basics of Semaphore. Here are some pointers for moving forward as you implement CI/CD pipelines for your projects:
- Explore the default Linux and macOS environments, which come with the most common open source build tools, languages, and databases.
- Install Semaphore CLI so you can easily debug jobs with SSH.
- Use secrets to authenticate with APIs and deploy to cloud environments.
- Use cache to reuse project dependencies.
- Study example projects.