CI/CD pipeline troubleshooting

Services and databases

Can't connect to service

Quick answer

Make sure you're using $WSR_SERVICE_HOST_[alias] when referring to the service host.

Related documentation

For a more in-depth explanation, see our guide on connecting to services in CI/CD.

Trouble connecting to a service (e.g., a PostgreSQL database) may be caused by using the wrong kind of hostname.

Take the following example…

./.gitlab-ci.yml

run-test-suite:
  stage: test
  services:
    - name: postgres:latest
      alias: my_psql_db
  variables:
    POSTGRES_DB: some_db
    POSTGRES_USER: user
    POSTGRES_PASSWORD: password
    POSTGRES_URI: postgresql://$POSTGRES_USER:$POSTGRES_PASSWORD@my_psql_db:5432/$POSTGRES_DB
  script:
    - psql "$POSTGRES_URI" --list

To fix it: instead of my_psql_db, use a Workshop-style reference like $WSR_SERVICE_HOST_my_psql_db.

./.gitlab-ci.yml

run-test-suite:
  stage: test
  services:
    - name: postgres:latest
      alias: my_psql_db
  variables:
    POSTGRES_DB: some_db
    POSTGRES_USER: user
    POSTGRES_PASSWORD: password
-   POSTGRES_URI: postgresql://$POSTGRES_USER:$POSTGRES_PASSWORD@my_psql_db:5432/$POSTGRES_DB
+   POSTGRES_URI: postgresql://$POSTGRES_USER:$POSTGRES_PASSWORD@$WSR_SERVICE_HOST_my_psql_db:5432/$POSTGRES_DB
  script:
    - psql "$POSTGRES_URI" --list

WSR_SERVICE_ value is missing

Quick answer

Escape the $ when referencing a WSR_SERVICE_ variable by itself: \$WSR_SERVICE_HOST_[alias].

Setting a variable to just the value of a WSR_SERVICE_ variable can result in an unexpectedely blank value.

Take the following example…

./.gitlab-ci.yml

run-test-suite:
  stage: test
  services:
    - name: postgres:latest
      alias: my_psql_db
  variables:
    POSTGRES_DB: some_db
    POSTGRES_USER: user
    POSTGRES_PASSWORD: password
    PGHOST: $WSR_SERVICE_HOST_my_psql_db
  script:
    - 'echo "Connecting to postgres at \"$PGHOST\""'

This example will output Connecting to postgres at "". This problem can often manifest as your code attempting to connect to databases on local unix sockets, for example:

dial error: dial unix /tmp/.s.PGSQL.5432: connect: no such file or directory

To fix it, escape the $ like this:

./.gitlab-ci.yml

run-test-suite:
  stage: test
  services:
    - name: postgres:latest
      alias: my_psql_db
  variables:
    POSTGRES_DB: some_db
    POSTGRES_USER: user
    POSTGRES_PASSWORD: password
-   PGHOST: $WSR_SERVICE_HOST_my_psql_db
+   PGHOST: \$WSR_SERVICE_HOST_my_psql_db
  script:
    - 'echo "Connecting to postgres at \"$PGHOST\""'

Out of memory

Quick answer

Add WORKER_MEMORY: xG to the job's variables: section.

Hitting out of memory errors, such as npm error 137 can be addressed by increasing the amount of memory that your job receives.

In the following example, the build job will be given 2 GB of memory:

./.gitlab-ci.yml

build:
  stage: build
  variables:
    WORKER_MEMORY: 2G
  script:
    - ...

Out of disk space

Quick answer

Add WORKER_DISK: xG to the job's variables: section

It can be tough to determine if your job runs out of disk space, but if you suspect that you might need more than the default 3 GB, you can specify up to 7 GB of disk for your jobs by adding WORKER_DISK to the job's variables: section:

./.gitlab-ci.yml

build:
  stage: build
  variables:
    WORKER_DISK: 7G
  script:
    - ...

Workshop Runners do not currently support more than 7GB of disk.