How to Install DeepSeek-R1 Locally with Docker

This tutorial provides a step-by-step guide to setting up and running DeepSeek-R1, a powerful AI model, using Docker containers. We’ll cover installing necessary software, configuring your environment, and finally deploying DeepSeek-R1 locally. Understanding Docker is crucial for consistent and efficient AI model deployment, ensuring that your model works reliably across different environments.

Before delving into the specifics of DeepSeek-R1, let's grasp the fundamental concept of Docker containers. Think of a Docker container as a self-contained package, a virtual environment that encapsulates everything an application needs to run: the code, runtime, system tools, libraries, and settings. This isolates the application from the host operating system, ensuring consistent behavior regardless of the underlying hardware or software configuration.

This is especially beneficial for AI models, which often have stringent dependencies and configuration requirements. A Docker container effectively packages the AI model and its environment, preventing compatibility issues that might arise when moving from a development machine to a server or cloud environment. It's like having a portable, self-contained "virtual machine" optimized for specific applications, guaranteeing consistent performance across various platforms.

Setting Up the Foundation: Installing Ubuntu and Docker

This guide assumes a familiarity with the command line. If you are not comfortable using a Linux terminal, we recommend looking up basic tutorials before proceeding. The instructions below are tailored for Windows users, though the Docker installation commands for other operating systems are readily available online.

Step 1: Installing Ubuntu on Windows (If Necessary)

For Windows users, accessing a Linux environment is streamlined through the Windows Subsystem for Linux (WSL). Follow these steps:

1. Open the Microsoft Store and search for "Ubuntu."

2. Select a version of Ubuntu and click "Get" to install it. This will download and install a complete Ubuntu Linux distribution within Windows.

3. Once installed, launch Ubuntu from your Start menu. You will be prompted to create a username and password for your new Linux environment.

4. Update your Ubuntu system: This step is crucial for ensuring you have the latest software packages and security updates. Open your Ubuntu terminal and execute the following command:

         sudo apt update && sudo apt upgrade
       

   This command first updates the package list (sudo apt update) and then upgrades all installed packages to their latest versions (sudo apt upgrade). The sudo command grants administrative privileges, which are needed for installing and managing software. The && symbol chains the two commands together, ensuring that the upgrade only occurs after the update is complete.

Step 2: Installing Docker

Next, we install Docker. Docker Engine is the core component that runs containers.

1. Verify Docker Installation: Open your Ubuntu terminal and check if Docker is already installed:

         docker --version
       

   If Docker is installed, you'll see the version number displayed. If not, proceed with the installation.

2. Install Docker: Use the following commands to install Docker:

   sudo apt update && sudo apt install docker.io -y
   sudo systemctl enable --now docker
       

   This sequence first updates the package list, then installs Docker (docker.io), using -y to automatically accept all prompts. Finally, it enables the Docker service so it starts automatically on boot and starts it immediately using systemctl enable --now docker.

Step 3: Prerequisites for NVIDIA GPU Acceleration (Optional)

If you have an NVIDIA GPU and want to leverage its processing power for DeepSeek-R1, you'll need the NVIDIA Container Toolkit. This toolkit allows Docker containers to utilize the GPU. Skip this step if you're not using a GPU.

1. Configure the NVIDIA Repository: Add the NVIDIA Container Toolkit repository to your system's package manager:

   curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
   && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
   sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
   sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
       

   This command securely adds the required repository to your system, enabling the installation of the toolkit. It uses curl to download the necessary files and gpg to verify their authenticity, preventing malicious installations. sed is used to modify the repository URL, adding a signature verification component for enhanced security.

2. Update the Package List: Refresh your system's package list to include the newly added repository:

         sudo apt-get update
       

3. Install the NVIDIA Container Toolkit: Install the toolkit:

         sudo apt-get install -y nvidia-container-toolkit
       

   This command installs all necessary components of the NVIDIA Container Toolkit, enabling GPU support within Docker containers.

Running Ollama and DeepSeek-R1 within Docker

Ollama is a platform for running large language models (LLMs). We'll use it to streamline the process of running DeepSeek-R1.

Step 4: Running Ollama in a Docker Container

This will download and run Ollama.

1. Run Ollama: Execute the following command in your Ubuntu terminal:

   docker run -d \
   --gpus all \  #Grants access to all GPUs if available. Comment out if not using a GPU
   -v ollama:/root/.ollama \ # Mounts a local directory named 'ollama' to the Ollama data directory inside the container
   -p 11434:11434 \ #Maps port 11434 on the host to port 11434 in the container
   --security-opt=no-new-privileges \ # Enhances security by limiting the container's privileges
   --cap-drop=ALL \ #Drops all capabilities except those explicitly added
   --cap-add=SYS_NICE \ # Allows the container to adjust process priorities
   --memory=8g \  # Allocates 8GB of RAM to the container
   --memory-swap=8g \ # Allows 8GB of swap space for the container
   --cpus=4 \ # Allocates 4 CPU cores to the container
   --read-only \ #Mounts the container's filesystem as read-only for better security
   --name ollama \ # Assigns the name "ollama" to the container for easier management
   ollama/ollama
       

   
   

   This command runs the ollama/ollama Docker image in detached mode (-d), allocating resources and setting security parameters. The -v flag creates a volume mount, persisting Ollama data even if the container is removed. The -p flag maps ports for external access. The flags --security-opt, --cap-drop, and --cap-add are crucial for enhancing security by limiting container privileges. The --memory, --memory-swap, and --cpus flags specify the resource limits for the container. Finally, --read-only and --name improve the security and manageability of the container.

Step 5: Running DeepSeek-R1 Locally

Once Ollama is running, you can use it to launch DeepSeek-R1:

1. Run DeepSeek-R1: Execute the following command within your Ubuntu terminal to start DeepSeek-R1 (replace 7b with the desired version if needed):

         docker exec -it ollama ollama run deepseek-r1:7b
       

   
   

   This command executes the ollama run deepseek-r1:7b command inside the already running ollama container. -it assigns a pseudo-TTY and keeps stdin open, letting you interact with DeepSeek-R1.

2. Interacting with DeepSeek-R1: Now you can interact with DeepSeek-R1. To exit the DeepSeek-R1 session, type /bye.

Step 6: Starting DeepSeek-R1 Subsequently

To start DeepSeek-R1 again, follow these steps:

1. Start the Ollama Container:

         docker start ollama
       

2. Run DeepSeek-R1:

         docker exec -it ollama ollama run deepseek-r1:7b
       

This comprehensive guide ensures a smooth and efficient setup of DeepSeek-R1 using Docker containers, maximizing performance and consistency across different environments. Remember to adapt resource allocation (--memory, --cpus) based on your system's capabilities.

Keywords: , LLM, Tutorial, Guide, Installation