Installation of Ollama

Ollama allows the local operation of Large Language Models (LLMs) on a Linux server. The models run on the own hardware and can be used via command line or HTTP-API.

This article presents the native installation of Ollama on an Ubuntu server. As an example, a small model is downloaded and tested by using the local command line as well as the local API.

Requirements

These instructions are designed for Ubuntu Server 24.04 LTS. The installation normally functions on an Ubuntu Server LTS and 22.04 LTS, too. For GPU-systems should be verified if a suitable driver is available for the used Ubuntu version in advance.

The following resources are recommended for initial testing:

• user with sudo-rights
• at least 8 GB RAM
• at least 20 GB storage available
• Internet access for installation and model download
• optional for a supported NVIDIA or AMD-GPU

Verify the intalled Ubuntu version:

cat /etc/os-release

Install curl:

sudo apt install -y curl

Install Ollama

Execute the official installation script:

curl -fsSL https://ollama.com/install.sh | sh

Next, verify the installed version:

ollama -v

The installation script sets up a systemd-service. Check its status:

sudo systemctl status ollama

The service should be displayed with the active (running) state.

If the service is not started, it can be activated manually:

sudo systemctl enable --now ollama

Start initial model

In this example, gemma3:4b is used. The model requires approximately 2.2 GB storage for the download and is suitable for the initial function test.

Start the model:

ollama run gemma3:4b

Ollama automatically downloads the model the first time it is runned. It may take a few minutes depending on the Internet connection.

Next, enter the following query, for example:

Explain the difference between RAID 5 and RAID 6.

Finish the session with:

/bye

Display the installed models:

ollama ls

Verify GPU-usage

Ollama automatically uses a supported GPU if the required driver is installed.

Start the model and open the second SSH-session:

ollama run gemma3:4b

Check the current version there:

ollama ps

The column PROCESSOR displays if the model is executed on the CPU, GPU or on both.

With an NVIDIA GPU, you can also check the utilization:

nvidia-smi

If there is no supported GPU available, Ollama executes the model via CPU and RAM. Response times are generally longer in such cases.

Check Ollama-API

Ollama provides a local HTTP-API on port 11434 by default.

Display the models available via API:

curl http://127.0.0.1:11434/api/tags

If the connection is successful, a JSON response containing the installed models is returned.

The local API is, for example, used by Open WebUI, n8n and own applications.

Optional: Acces from the local network

By default, Ollama only listens on 127.0.0.1. If another system should access the API in the local network, the Bind address must be adjusted.

Open the systemd extension:

sudo systemctl edit ollama.service

Add the following content:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Apply changes:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Next, verify the port:

sudo ss -tulpn | grep 11434

By default, the Ollama API does not have its own user authentication for locally run models. Port 11434 should therefore only be released for reuqired systems and should not be directly available from the Internet.

Here is an example for the release of an internal subnet with UFW:

sudo ufw allow from 192.168.1.0/24 to any port 11434 proto tcp

Customize the subnet to suit your own environment.

Further Ollama environment variables can be added in the same systemd-extension. All Environment="..." lines are entered together under a [Service] block.

Update Ollama

On Linux, Ollama is updated by running the installation script again:

curl -fsSL https://ollama.com/install.sh | sh

Models that have already been downloaded will remain.

After that, check the version:

ollama -v

Troubleshooting

Ollama-service does not run

Verify the service status:

sudo systemctl status ollama

Display the most recent log entries:

sudo journalctl -e -u ollama

Model does not start

Check the available storage space:

df -h

Display the installed models:

ollama ls

If necessary, download the model again:

ollama pull gemma3:4b

Port 11434 is not available

Check if Ollama listens on the port:

sudo ss -tulpn | grep 11434

Restart the service:

sudo systemctl restart ollama

Sources

• Ollama Documentation – Linux (en)
• Ollama Documentation – Quickstart (en)
• Ollama Documentation – API Introduction (en)
• Ollama Documentation – FAQ (en)
• Ollama Model Library – Gemma 3 4B (en)
• Ubuntu Release Cycle (en)

Author: Florian Müller

Translator: Alina Ranzinger Alina has been working at Thomas-Krenn.AG since 2024. After her training as multilingual business assistant, she got her job as assistant of the Product Management and is responsible for the translation of texts and for the organisation of the department.