Connect to a service in CI/CD

Purpose of this guide

This guide is meant to explain the different ways you might connect with services in Workshop's CI/CD, with particular regard to the ways Workshop may differ from GitLab.com.

Services explained

If you're coming from another CI/CD platform, you may be used to running what we call "services" (e.g., a database) through something like Docker, Docker Compose, and perhaps Docker in Docker (DinD). GitLab supports a similar approach with service images, though services are defined in addition to, rather than alongside, your main job image. Workshop does not currenlty support Docker in Docker.

Further information on services can be found in GitLab documentation for:

Service section of GitLab's CI/CD YAML syntax reference: a base reference for the service syntax expected in .gitlab-ci.yml files.
GitLab's CI/CD job services guide: walks through defining services more broadly, including common tasks, pitfalls and examples for popular databases.

Defining the service

The service key defines a list of maps for each service, each usually given keys for at least:

name: what image the service will use, e.g., postgres:15
alias: token by which the service will be referenced (can be omitted if not referencing the service)

The following demonstrates how you might connect to a PostgreSQL database running as a service alongisde a test job.

Note in particular how, on line #6, the service is given an arbitrary alias of my_psql_db. That alias is then prefixed with $WSR_SERVICE_HOST_ to reference the service's host name on line #10; an important difference from GitLab.com.

./.gitlab-ci.yml
# ...
run-test-suite:
  stage: test
  services:
    - name: postgres:15
      alias: my_psql_db
  variables:
    PG_USER: user
    PG_PASS: password
    PG_URI: postgresql://$PG_USER:$PG_PASS@$WSR_SERVICE_HOST_my_psql_db:5432/some_db
  script:
    - psql "$PG_URI"

Using service references

Unlike GitLab.com, which uses an isolated virtual network, Workshop sets unpredictable hostnames for service containers that are meant to prevent collisions within a shared namespace. This means you cannot know your service's hostname in advance — to overcome this limitation we provide dynamically defined environment variables as described in the table below. These variables refer to ephemeral CloudFoundry apps, each a container deployed with the service image.

Variable prefix	Usage example	Description
`WSR_SERVICE_ID_`	`$WSR_SERVICE_ID_my_alias`	CloudFoundry app ID
`WSR_SERVICE_HOST_`	`$WSR_SERVICE_HOST_my_alias`	CloudFoundry app hostname

Using refs in…

✅ `[job].variables`

Your jobs’ variables blocks are probably the most common place you'll use service references.

./.gitlab-ci.yml
# ...
run-test-suite:
  stage: test
  services:
    - name: postgres:15
      alias: my_psql_db
  variables:
    PG_USER: user
    PG_PASS: password
    PG_URI: postgresql://$PG_USER:$PG_PASS@$WSR_SERVICE_HOST_my_psql_db:5432/some_db
  script:
    - psql "$PG_URI"

✅ `[job].script`

You can also reference services directly from script steps.

./.gitlab-ci.yml
# ...
run-test-suite:
  stage: test
  services:
    - name: postgres:15
      alias: my_psql_db
  variables:
    PG_USER: user
    PG_PASS: password
  script:
    - psql "postgresql://$PG_USER:$PG_PASS@$WSR_SERVICE_HOST_my_psql_db:5432/some_db"

✅ `[job].before_script`

In some circumstances you may want to use service references in a before_script. Here’s an example where we dynamically define a variable on all jobs by adding it to the default.before_script.

./.gitlab-ci.yml
# ...
default:
  services:
    - name: postgres:15
      alias: my_psql_db
  before_script:
    - export DB_URI="postgresql://$DYN_USER:$DYN_PASS@$WSR_SERVICE_HOST_my_psql_db:5432/$DYN_DB"

❌ `[job].services.0.variables`

danger

Service references will not work within services[...].variables. Check tracking issue workshop/runner/gitlab-runner-cloudgov#176 for the most current information.

The main exception is within the services' own variables block, i.e., services[...].variables. Variables within that block are not currently interpreted due to, in part, an upstream bug in GitLab's runner. GitLab's runner ought to interpolate the variables defined here as it does elsewhere, but currently does not — we in turn leave these variables alone to keep behavior consistent.

Ergo, the below configuration will not work.

./.gitlab-ci.yml
# ...
run-test-suite:
  stage: test
  services:
    - name: postgres:15
      alias: my_missing_db
    - name: custom_app:latest
      variables:
        DB_URI: "failql://$PG_USER:$PG_PASS@$WSR_SERVICE_HOST_my_missing_db:5432/broke_db"
  variables:
    PG_USER: user
    PG_PASS: password

Purpose of this guide​

Services explained​

Defining the service​

Using service references​

Using refs in…​

✅ [job].variables​

✅ [job].script​

✅ [job].before_script​

❌ [job].services.0.variables​