Setting up Python environment for GitLab runner with shell

During configuration of a runner there a several executors to choose from. This documents explains how to setup a Python environment for the shell executor.

Prepare the GitLab runner server

sudo apt install python3.10-venv
sudo mkdir /var/gitlab-runner-venvs
sudo chown gitlab-runner:gitlab-runner /var/gitlab-runner-venvs
sudo chown -R gitlab-runner:gitlab-runner /var/lib/gitlab-runner

You can now add a new runner to your gitlab-runner server (make sure to select shell as the executor) or modify an existing runner from docker. Change the docker executor to the shell executor in /etc/gitlab-runner/config.toml:

...
executor = "shell"
...

Note: the ... indicate other settings

Setup the CI/CD script

stages:
  - setup
  - build
  - deploy

variables:
  VENV_PATH: /var/gitlab-runner-venvs/<projectname>

setup:
  stage: setup
  script:
    - if ! [[ -d "$VENV_PATH" ]]; then echo "VENV Directory $VENV_PATH does not exist. Creating it..."; python3 -m venv "$VENV_PATH"; echo "Directory $VENV_PATH created."; fi
    - source $VENV_PATH/bin/activate
    - if [[ -e requirements.txt ]]; then echo "Updating packages..."; pip install -r requirements.txt --upgrade; echo "Packages updated."; fi
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      changes:
        - requirements.txt

build:
  stage: build
  script:
  - source $VENV_PATH/bin/activate
  - mkdocs build
  artifacts:
    paths:
      - site
    expire_in: 1 week

deploy:
  stage: deploy
  script:
  - rsync -rv --delete site/ <destination>

Post actions

After changing the .gitlab-ci.yml and pushing it to the gitlab server you will see an error in the pipeline. This is because the CI/CD is still using the previous jobs-structure and not the new one. You can solve this by manually rerunning the pipeline after this failure.

---

Last update: 2024-01-31