GitLab CI

With its hosted CI/CD Service, GitLab offers developers "a tool built into GitLab for software development through the continuous methodologies".

Detailed documentation is available in the GitLab CI/CD Documentation.

Basic Setup

The example below is basic CI setup and job using GitLab CI/CD to run Cypress tests within the Electron browser. This GitLab CI configuration is placed within .gitlab-ci.yml.

stages:
  - test

test:
  image: node:latest
  stage: test
  script:
    # install dependencies
    - npm ci
    # start the server in the background
    - npm run start:ci &
    # run Cypress tests
    - npm run e2e

How this configuration works:

• On push to this repository, this job will provision and start GitLab-hosted Linux instance for running the outlined stages declared in script with in the test job section of the configuration.
• The code is checked out from our GitHub/GitLab repository.
• Finally, our scripts will:
  • Install npm dependencies
  • Start the project web server (npm start)
  • Run the Cypress tests within our GitHub repository within Electron.

Testing in Chrome and Firefox with Cypress Docker Images

The Cypress team maintains the official Docker Images for running Cypress tests locally and in CI, which are built with Google Chrome and Firefox. For example, this allows us to run the tests in Firefox by passing the --browser firefox attribute to cypress run.

stages:
  - test

test:
  image: cypress/browsers:node12.14.1-chrome85-ff81
  stage: test
  script:
    # install dependencies
    - npm ci
    # start the server in the background
    - npm run start:ci &
    # run Cypress tests
    - npx cypress run --browser firefox

Caching Dependencies and Build Artifacts

Caching of dependencies and build artifacts can be accomplished with the cache configuration. The caching documentation contains all options for caching dependencies and build artifacts across many different workflows. Artifacts from a job can be defined by providing paths and an optional expiry time.

stages:
  - test

cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths:
    - node_modules/
    - .npm/

test:
  image: cypress/browsers:node12.14.1-chrome85-ff81
  stage: test
  script:
    # install dependencies
    - npm ci
    # start the server in the background
    - npm run start:ci &
    # run Cypress tests
    - npx cypress run --browser firefox
  artifacts:
    when: always
    paths:
      - cypress/videos/**/*.mp4
      - cypress/screenshots/**/*.png
    expire_in: 1 day

Parallelization

The Cypress Dashboard offers the ability to parallelize and group test runs along with additional insights and analytics for Cypress tests.

The addition of the parallel attribute to the configuration of a job will allow us to run multiples instances of Cypress at same time as we will see later in this section.

Before diving into an example of a parallelization setup, it is important to understand the two different types of jobs that we will declare:

• Install Job: A job that installs and caches dependencies that will be used by subsequent jobs later in the GitLab CI workflow.
• Worker Job: A job that handles execution of Cypress tests and depends on the install job.

Install Job

The separation of installation from test running is necessary when running parallel jobs. It allows for reuse of various build steps aided by caching.

First, we will define the build stage along with cache and variables related to the cache.

Then we define the install step that will be used by the worker jobs and assign it to the build stage.

stages:
  - build

## Set environment variables for folders in "cache" job settings for npm modules and Cypress binary
variables:
  npm_config_cache: '$CI_PROJECT_DIR/.npm'
  CYPRESS_CACHE_FOLDER: '$CI_PROJECT_DIR/cache/Cypress'

cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths:
    - cache/Cypress
    - node_modules
    - build

## Install NPM dependencies and Cypress
install:
  image: cypress/browsers:node12.14.1-chrome85-ff81
  stage: build
  script:
    - npm ci

Worker Jobs

Next, we add a test stage and define the worker job named ui-chrome-tests that will run Cypress tests with Chrome in parallel during the test stage.

The addition of the parallel attribute to the configuration of a job will allow us to run multiples instances of Cypress at same time.

stages:
  - build
  - test

## Set environment variables for folders in "cache" job settings for npm modules and Cypress binary
variables:
  npm_config_cache: '$CI_PROJECT_DIR/.npm'
  CYPRESS_CACHE_FOLDER: '$CI_PROJECT_DIR/cache/Cypress'

cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths:
    - .cache/*
    - cache/Cypress
    - node_modules
    - build

## Install NPM dependencies and Cypress
install:
  image: cypress/browsers:node12.14.1-chrome85-ff81
  stage: build
  script:
    - npm ci

ui-chrome-tests:
  image: cypress/browsers:node12.14.1-chrome85-ff81
  stage: test
  parallel: 5
  script:
    # install dependencies
    - npm ci
    # start the server in the background
    - npm run start:ci &
    # run Cypress tests in parallel
    - npx cypress run --record --parallel --browser chrome --group "UI - Chrome"

Using the Cypress Dashboard with GitLab CI/CD

In the GitLab CI configuration we have defined in the previous section, we are leveraging three useful features of the Cypress Dashboard:

1. Recording test results with the --record flag to the Cypress Dashboard:

   • In-depth and shareable test reports.
   • Visibility into test failures via quick access to error messages, stack traces, screenshots, videos, and contextual details.
   • Integrating testing with the merge-request process via commit status guards and convenient test report comments.
   • Detecting flaky tests and surfacing them via Slack alerts or GitLab PR status checks.

2. Parallelizing test runs and optimizing their execution via intelligent load-balancing of test specs across CI machines with the --parallel flag.

3. Organizing and consolidating multiple cypress run calls by labeled groups into a single report within the. Cypress Dashboard. In the example above we use the --group "UI - Chrome" flag to organize all UI tests for the Chrome browser into a group labeled "UI - Chrome" in the Cypress Dashboard report.

Cypress Real World Example with GitLab CI/CD

A complete CI workflow against multiple browsers, viewports and operating systems is available in the Real World App (RWA).

Clone the Real World App (RWA) and refer to the .gitlab-ci.yml file.