Jupyter¶
Notebook server¶
It is possible to run the Python kernel of a Jupyter notebook on a server (e.g. HPC) and remotely access this notebook on your desktop or laptop (the client) via a webbrowser.
Note
In this manual we presume you are using a Linux server.
Note
Most instructions given here are found here: https://jupyter-notebook.readthedocs.io/en/stable/public_server.html. For more information check this webpage.
Setup Miniconda¶
First you need to install Miniconda in your homedirectory on the server: Miniconda.
Then create a new virtual environment in Miniconda and install the Jupyter-package.
$ conda create -n notebook_server jupyter
Note
In this example we choose the virtual environment name to be notebook_server. Also note that with this command after creation the Jupyter-package is installed
Setup notebook server¶
First activate the virtual environment and create the Jupyter configuration file:
$ conda activate notebook_server
$ jupyter notebook --generate-config
Create a password to secure access to your notebook. This password will be requested when opening the website to your notebook:
$ jupyter notebook password
To encrypt your connection over the internet setup an SSL certificate which will be used when starting up the notebook server:
$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout notebook_server.key -out notebook_server.pem
Startup notebook server¶
You are now ready to start the notebook server with the SSL certificate:
$ jupyter notebook --certfile=notebook_server.pem --keyfile notebook_server.key
The server will be started and show you the log information on the terminal. The webinterface (the actual Jupyter Notebook) is accessed via port 8888. If you are ready using the notebook server you can quit the server in the terminal with Ctrl+C.
Notebook client¶
Many thanks to Martijn Nagtegaal
You can now access this Jupyter Notebook via your desktop or laptop (the client). Most HPC servers are behind a firewall and will not allow you to directly access the server via port 8888. This can be solved by opening up a SSH-tunnel between the server and your client which will use the ssh-connection over port 22 to pass through the otherwise blocked traffic to port 8888 (see the diagram below). Almost all servers will allow ssh-connections over port 22.
The data transport from the server on port 8888 is tunneled to the client on port 18888. This enables the user to access the remote port 8888 via the local port 18888. You can then access the notebook on the browser on your desktop or laptop via the URL https://localhost:18888.
In Linux or in Windows with WSL this tunnel is created on the client as follows:
$ ssh -L 18888:localhost:8888 <server ip name>
Note
The tunnel will remain open as long as your ssh-connection is up.
In Windows you can also create a tunnel with PuTTY (see Remote login with SSH). For more information check:
Check connection¶
You can easily check if your Jupyter client is actually using the Jupyter server with the following code:
import socket
print(socket.gethostname())
This should give you the hostname of the Jupyter server