Add GitLab Runners

Add GitLab Runners to the PHX reference implementation, including an optional air-gapped simulation.
Categories:
Tags:

Projects:  c2platform/phx/ansible ,  c2platform.mgmt.gitlab runner

Overview

This how-to describes how to create GitLab Runners on nodes pxd-runner1 and pxd-runner2 using the Ansible GitLab Runner play. Both nodes are configured similarly, but pxd-runner2 simulates an air-gapped installation based on a manual installation variant. This allows testing scenarios where internet access is restricted, such as downloading packages offline.

GitLab Runners execute CI/CD pipelines for your GitLab projects. In this setup, we provision runners with different executors (e.g., Docker and Shell) to handle various job types securely and efficiently.

Prerequisites

  • Setting Up GitLab CE: This guide describes how to provision and configure a GitLab Community Edition instance in the PHX reference implementation.
  • Ensure the nodes pxd-rproxy1, pxd-ad, and pxd-gitlab are up and running.

Setup

To create the runner on the pxd-runner1 node and register it, run the following command. This takes around 4 minutes to complete:

vagrant up pxd-runner1

Air-Gapped Installation (Optional)

To perform an installation that simulates an air-gapped environment (based on the manual installation variant), create the pxd-runner2 node. This takes around 15 minutes to complete:

vagrant up pxd-runner2

Verify

Runners

Navigate to Groups → C2 Platform → Build → Runners  . Depending on whether you provisioned pxd-runner2, you should see 4 or 8 registered runners. Without pxd-runner2, there are 4 runners (one each for Docker and Shell executors in VM and container modes). With pxd-runner2, this doubles to 8 runners. The image below shows an example with 4 runners.

Git LFS and GitLab Pages

Navigate to Groups → C2 Platform → C2 Platform → Examples → Git LFS and GitLab Pages → Build → Pipelines  and start a new pipeline.

Once the pipeline completes, navigate to Deploy → Pages  . This page displays the URL to the GitLab Pages website created by the pipeline.

Click on the URL to access the landing page for the Ansible Software Repository.

If you click on the link for apache-tomcat-10.1.19.tar.gz, it should download the tarball. This demonstrates that the GitLab Runner successfully executed the pipeline, including publishing a GitLab Pages website that implements a software repository using Git LFS and GitLab Pages.

GitLab Runner CLI

SSH into pxd-runner1:

vagrant ssh pxd-runner1

Switch to root:

sudo su -

You can now use the GitLab Runner CLI to perform various tasks. For help:

gitlab-runner --help
Show me
root@pxd-runner1:~# gitlab-runner --help
NAME:
   gitlab-runner - a GitLab Runner

USAGE:
   gitlab-runner [global options] command [command options] [arguments...]

VERSION:
   18.9.0 (07e534ba)

AUTHOR:
   GitLab Inc. <support@gitlab.com>

COMMANDS:
   list                  List all configured runners
   register              register a new runner
   reset-token           reset a runner's token
   run                   run multi runner service
   run-single            start single runner
   wrapper               start multi runner service wrapped with gRPC manager server
   unregister            unregister specific runner
   verify                verify all registered runners
   fleeting              manage fleeting plugins
   artifacts-downloader  download and extract build artifacts (internal)
   artifacts-uploader    create and upload build artifacts (internal)
   cache-archiver        create and upload cache artifacts (internal)
   cache-extractor       download and extract cache artifacts (internal)
   cache-init            changed permissions for cache paths (internal)
   health-check          check health for a specific address
   proxy-exec            execute internal commands (internal)
   read-logs             reads job logs from a file, used by kubernetes executor (internal)
   install               install service
   uninstall             uninstall service
   start                 start service
   stop                  stop service
   restart               restart service
   status                get status of a service
   help, h               Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --cpuprofile value           write cpu profile to file [$CPU_PROFILE]
   --debug                      debug mode [$RUNNER_DEBUG]
   --log-format value           Choose log format (options: runner, text, json) [$LOG_FORMAT]
   --log-level value, -l value  Log level (options: debug, info, warn, error, fatal, panic) [$LOG_LEVEL]
   --help, -h                   show help
   --version, -v                print the version

For example, list the runners on this node:

gitlab-runner list

root@pxd-runner1:~# gitlab-runner list
Runtime platform                                    arch=amd64 os=linux pid=4060 revision=dcfb4b66 version=15.10.1
Listing configured runners                          ConfigFile=/etc/gitlab-runner/config.toml
pxd-runner1                                   Executor=shell Token=btPy-VV2HF2DxVxh1xzM URL=https://gitlab.com/
root@pxd-runner1:~#

Review

Vagrant Boxes

The Vagrant configuration defines two runner nodes: pxd-runner1 and pxd-runner2. pxd-runner2 includes an additional label for air-gapped simulation. These boxes use Ubuntu 24 LXD and apply the mgmt/gitlab_runner play.

Box definition in Vagrantfile.yml:

 Vagrantfile.yml

253  - name: runner1
254    short_description: Gitlab Runner
255    description: Gitlab Runner
256    box: ubuntu24-lxd
257    ip-address: 192.168.60.13
258    plays:
259      - mgmt/gitlab_runner
260    labels:
261      - gitlab_runner
262  - name: runner2
263    short_description: Gitlab Runner
264    description: Gitlab Runner
265    box: ubuntu24-lxd
266    ip-address: 192.168.60.15
267    plays:
268      - mgmt/gitlab_runner
269    labels:
270      - gitlab_runner
271      - gitlab_runner_air_gapped

Play

The Ansible play targets hosts labeled gitlab_runner or gitlab_runner_air_gapped. It applies core Linux roles and the c2platform.mgmt.gitlab_runner role to set up the runners.

 plays/mgmt/gitlab_runner.yml

---
- name: GitLab Runner
  hosts: gitlab_runner:gitlab_runner_air_gapped
  become: true

  roles:
    - c2platform.core.linux
    - c2platform.mgmt.gitlab_runner

Main Configuration

The main configuration for the GitLab Runner is in group_vars/gitlab_runner/main.yml. It defines the runner version, coordinator URL, authentication token, packages to install, group ID, environment variables (for certificate trust), and disables default runner creation. It also sets up custom resources like user groups and configuration files for concurrency and logging.

 group_vars/gitlab_runner/main.yml

---
px_gitlab_runner_version: 18.9.0-1
gitlab_runner_coordinator_url: "https://{{ px_gitlab_domain }}"
gitlab_runner_token: "{{ px_gitlab_root_pat }}"
gitlab_runner_package:
  - "gitlab-runner-helper-images={{ px_gitlab_runner_version }}"
  - "gitlab-runner={{ px_gitlab_runner_version }}"
gitlab_runner_group: 2  # group: c2platform  # TODO PHX-548

gitlab_runner_environment:
  REQUESTS_CA_BUNDLE: "{{ px_linux_cert_dir }}/c2.crt.crt"

# Disable default runner creation
gitlab_runner_resource_groups_disabled: [default]

gitlab_runner_resources:
  0_config:
    - name: gitlab-runner  # required for vm-runner and shell-exe
      module: user
      groups: docker
      append: true
    - name: config.toml & config_template.toml
      module: copy
      defaults:
        mode: '644'
      resources:
        - dest: /etc/gitlab-runner/config.toml
          content: |
            concurrent = 1
            check_interval = 0
            shutdown_timeout = 0
            log_format = "text"
          force: false
        - dest: /etc/gitlab-runner/config_template.toml
          content: |
            [[runners]]
              [runners.docker]
                host = "unix:///var/run/docker.sock"
                volumes = ["/var/run/docker.sock:/var/run/docker.sock"]
                privileged = false

Custom GitLab Runner Image

This configuration builds a custom Docker image for the GitLab Runner that trusts C2 certificates. It creates a directory, copies certificates and configs, updates the Dockerfile with the actual Docker GID, and builds the image (gitlab/c2-gitlab-runner:latest). This ensures runners in container mode can handle Docker operations securely with proper certificate trust.

 group_vars/gitlab_runner/image.yml

---
# GitLab Runner config and GitLab Runner image that trusts C2 certificates
gitlab_runner_resources:
  0_gitlab_runner_image:
    - name: /opt/gitlab-runner
      module: file
      path: /opt/gitlab-runner
      mode: '755'
      state: directory
    - name: /opt/gitlab-runner/Dockerfile
      module: copy
      defaults:
        mode: '644'
      resources:
        - dest: /opt/gitlab-runner/c2.crt
          src: "{{ px_linux_cert_dir }}/c2.crt.crt"
          remote_src: true
        - dest: /opt/gitlab-runner/config.toml
          src: /etc/gitlab-runner/config.toml
          remote_src: true
        - dest: /opt/gitlab-runner/config_template.toml
          src: /etc/gitlab-runner/config_template.toml
          remote_src: true
        - dest: /opt/gitlab-runner/Dockerfile.tpl
          content: |
            # for container-runner with shell-exe:
            # - docker.io package required
            # - gitlab-runner in docker group
            FROM gitlab/gitlab-runner:latest
            ARG DOCKER_GID=999
            COPY c2.crt /usr/local/share/ca-certificates/c2.crt
            COPY config.toml /etc/gitlab-runner/config.toml
            COPY config_template.toml /etc/gitlab-runner/config_template.toml
            RUN update-ca-certificates \
              && apt update \
              && apt install docker.io -y \
              && apt clean
            RUN if ! getent group docker; then \
                  groupadd -g ${DOCKER_GID} docker; \
                else \
                  groupmod -g ${DOCKER_GID} docker; \
                fi
            RUN usermod -a -G docker gitlab-runner
    - name: Update Dockerfile with actual Docker GID
      module: shell
      cmd: |
        cat /opt/gitlab-runner/Dockerfile.tpl > /opt/gitlab-runner/Dockerfile
        DOCKER_GID=$(getent group docker | cut -d: -f3)
        sed -i "s/ARG DOCKER_GID=999/ARG DOCKER_GID=${DOCKER_GID}/" \
          /opt/gitlab-runner/Dockerfile
    - name: gitlab/c2-gitlab-runner
      module: docker_image
      tag: latest
      build:
        path: /opt/gitlab-runner
        dockerfile: Dockerfile
        pull: true
      source: build
      state: present

Runners Configuration

This configuration registers multiple runners with specific tags, executors, and settings. It defines tags based on the environment (e.g., hostname, OS), and registers runners for Docker and Shell executors in both VM and container modes. The common registration flags ensure non-interactive setup with the coordinator URL.

 group_vars/gitlab_runner/runners.yml

---
gitlab_runner_tag_list:
  - px
  - pxd
  - "{{ ansible_fqdn }}"
  - "{{ ansible_distribution | lower }}"
  - "{{ ansible_os_family | lower }}"
  - "{{ ansible_distribution_version }}"

gitlab_runner_resources:
  1_runners:
    - name: Register runners
      module: gitlab_runner
      defaults:
        api_url: "{{ gitlab_runner_coordinator_url }}"
        api_token: "{{ gitlab_runner_token }}"
        environment: "{{ gitlab_runner_environment }}"
        active: true
        run_untagged: true
        locked: false
        group: "{{ gitlab_runner_group }}"
      resources:
        - description: "{{ inventory_hostname }}-docker"
          tag_list: "{{ gitlab_runner_tag_list + ['vm-runner', 'docker-exe'] }}"
          register: >-
            {{ px_gitlab_runner_register_common }}
            --executor "docker"
            --docker-image alpine:latest
            --template-config /etc/gitlab-runner/config_template.toml
        - description: "{{ inventory_hostname }}-shell"
          tag_list: "{{ gitlab_runner_tag_list + ['vm-runner', 'shell-exe'] }}"
          register: >-
            {{ px_gitlab_runner_register_common }}
            --executor "shell"
        - description: "{{ inventory_hostname }}-dind"
          tag_list: "{{ gitlab_runner_tag_list + ['container-runner', 'docker-exe'] }}"
          container:
            name: "gitlab-runner-{{ inventory_hostname }}-docker"
            image: gitlab/c2-gitlab-runner:latest
          register: >-
            {{ px_gitlab_runner_register_common }}
            --executor "docker"
            --docker-image alpine:latest
            --template-config /etc/gitlab-runner/config_template.toml
        - description: "{{ inventory_hostname }}-sind"
          tag_list: "{{ gitlab_runner_tag_list + ['container-runner', 'shell-exe'] }}"
          container:
            name: "gitlab-runner-{{ inventory_hostname }}-shell"
            image: gitlab/c2-gitlab-runner:latest
          register: >-
            {{ px_gitlab_runner_register_common }}
            --executor "shell"

px_gitlab_runner_register_common: >-
  --non-interactive
  --url "{{ gitlab_runner_coordinator_url }}"

Air-Gapped Installation

The air-gapped configuration for pxd-runner2 disables the default installation from the GitLab Runner role and replaces it with a custom installation method suitable for environments without internet access. This is defined in group_vars/gitlab_runner_air_gapped/main.yml. It disables the 1_install resource group and introduces 1_install_air_gapped, which installs DEB packages from a local repository, enables the service, and sets the version and repository URLs accordingly. This demonstrates the flexibility of the Ansible role in supporting custom, offline installation variants.

 group_vars/gitlab_runner_air_gapped/main.yml

---
# Disable default install from GitLab Runner role
gitlab_runner_bootstrap_resource_groups_disabled: [1_install]

# Configure custom install
gitlab_runner_bootstrap_resources:
  1_install_air_gapped:
    - name: gitlab-runner-helper-images.deb
      module: apt
      deb: "{{ px_gitlab_runner_repo }}/gitlab-runner-helper-images.deb"
    - name: gitlab-runner_amd64.deb
      module: apt
      deb: "{{ px_gitlab_runner_repo }}/gitlab-runner_amd64.deb"
    - name: gitlab-runner
      module: service
      enabled: true

gitlab_runner_package_urls:
  Debian: "{{ px_gitlab_runner_repo }}/gitlab-runner_amd64.deb"

# https://gitlab.com/gitlab-org/gitlab-runner/-/releases
px_gitlab_runner_version: v18.9.0
# px_gitlab_runner_version: latest
px_gitlab_runner_repo: >-
  {{ "https://s3.dualstack.us-east-1.amazonaws.com/gitlab-runner-downloads/"
  ~ px_gitlab_runner_version ~ "/deb"
  }}

See also default install in roles/gitlab_runner/defaults/main.yml in GitLab Runner role c2platform.mgmt.gitlab_runner:

 roles/gitlab_runner/defaults/main.yml

37  1_install:
38    - name: Download GitLab Runner repository installation script
39      module: get_url
40      url: "{{ gitlab_runner_repository_installation_script_url }}"
41      dest: "{{ gitlab_runner_repo_script_path }}"
42    - name: Install GitLab Runner repository
43      module: command
44      cmd: "bash {{ gitlab_runner_repo_script_path }}"
45      changed_when: true
46      # creates TODO
47    - name: "{{ gitlab_runner_package }}"
48      module: package
49      state: present
50    - name: gitlab-runner
51      module: service
52      enabled: true

Additional Information

For further reference, explore the following information:


Last modified March 31, 2026: howto phx gitlab runners pxd-gitlab pxd-runner1 pxd-runner2 PHX-540 (01b3402)