Skip to content

MkDocs automatic website generation

This guide outlines the process for setting up and maintaining documentation using MkDocs, hosted within a GitLab project. MkDocs allows for easy management of documentation, which is automatically compiled into a static website upon each update to the repository. The automation is facilitated through GitLab’s CI/CD features, leveraging a .gitlab-ci.yml configuration file for specifying build commands.

Repository and CI/CD Configuration

The source code and documentation setup are hosted on GitLab. You can find the project here:

The CI/CD pipeline automates the generation and deployment of the static website. This automation relies on two servers: a runner-server to execute CI/CD scripts and a web-server to serve the generated website.

Server Requirements

  • runner-server (gitlab-runner): gitlab-imphys.tnw.tudelft.nl
    • Requires the gitlab-runner daemon installed.
  • web-server (apache): qiweb.tudelft.nl
    • Requires Apache web server.

Setting up servers

These instructions are only needed if you haven’t already setup the runner-server and the web-server. Note that you can also decide to implement both runner-service and web-service on one server. In this setup two separate servers are used.

Both servers are virtual servers supplied by the ICT department of the TU. You can read about these virtual servers here:

and use this form to request them:

Make sure to select the Basic Ubuntu configuration and for the web-server open firewall port 80 and 443 (http and https). Note that per default firewall 22 (ssh) is open for access from the bastion hosts (linux-bastion.tudelft.nl, linux-bastion-ex.tudelft.nl and student-linux.tudelft.nl).

After release of the servers make sure you harden them for network access. At least turn on the firewall (ufw) and install fail2ban to block unwanted access. The configuration of these tools is outside the scope of this document.

runner-server

Follow these instructions to install gitlab-runner:

Later in this document the configuration will be explained.

web-server

Install apache2:

apt install apache2

Later in this document the configuration will be explained.

Runner Configuration

To link the runner server with the GitLab project:

  1. Navigate to Settings > CI/CD > Runners > Project runners > New project runner in GitLab.
  2. Configure with the following settings:
    • OS: Linux
    • Allow running untaged jobs.
    • Runner description: sysman documentation

  3. Follow the instructions for registering gitlab-runner on the runner server with the provided url and token:

    sudo gitlab-runner register \
  --url https://gitlab.tudelft.nl \
  --token [TOKEN]

    Use the following settings:

    • URL: the default value entered in the command
    • name for the runner: sysman documentation
    • executor: shell

The runner needs to setup a Python virtual environment for execution of mkdocs (a Python package). This is done in the CI/CD script which will create a directory in /var/gitlab-runner-venvs with the name sysman (see VENV_PATH in the script).

Web Server Setup

Note: for this configuration we assume apache runs with the web account

All MkDocs generated webpages will be copied to the home-directory of the web-user: /home/web.

The gitlab-runner user on the runner-server must have SSH access to the web server for deployment. This requires generating an SSH keypair on the runner-server and adding the public key to the web-server’s authorized keys for the user web.

Runner Server SSH Setup

sudo -u gitlab-runner bash -
mkdir ~/.ssh
cd ~/.ssh
ssh-keygen -t rsa -b 4096 # No passphrase

Copy the content of ~/.ssh/id_rsa.pub.

Web Server SSH Authorization

sudo -u web bash -
mkdir ~/.ssh
cd ~/.ssh
echo <content of public key> >> authorized_keys

CI/CD Pipeline

The .gitlab-ci.yml file dictates the pipeline for website creation and deployment. This includes the setup of a Python virtual environment for MkDocs, the rendering of the website with mkdocs and the deployment process using rsync over SSH. Note that the shell method is used to execute the commands for creating the website. This allows for easy inclusion or modification of the used Python packages.

Read more information about using the shell method here.

Final Steps on Web Server

After the CI/CD pipeline runs for the first time, link the deployed site to the web server’s document root to make it accessible:

sudo ln -s /home/web/sysman /var/www/html/sysman
Last update: 2024-07-22