This tutorial provides a step-by-step guide to setting up and running DeepSeek-R1, a powerful AI model, using Docker containers. We’ll cover installing necessary software, configuring your environment, and finally deploying DeepSeek-R1 locally. Understanding Docker is crucial for consistent and efficient AI model deployment, ensuring that your model works reliably across different environments.
Before delving into the specifics of DeepSeek-R1, let's grasp the fundamental concept of Docker containers. Think of a Docker container as a self-contained package, a virtual environment that encapsulates everything an application needs to run: the code, runtime, system tools, libraries, and settings. This isolates the application from the host operating system, ensuring consistent behavior regardless of the underlying hardware or software configuration.
Open the Microsoft Store and search for "Ubuntu." Select a version of Ubuntu and click "Get" to install it. This will download and install a complete Ubuntu Linux distribution within Windows. Once installed, launch Ubuntu from your Start menu. You will be prompted to create a username and password for your new Linux environment. Update your Ubuntu system: This step is crucial for ensuring you have the latest software packages and security updates. Open your Ubuntu terminal and execute the following command:sudo apt update && sudo apt upgrade
This command first updates the package list (sudo apt update) and then upgrades all installed packages to their latest versions (sudo apt upgrade). The sudo command grants administrative privileges, which are needed for installing and managing software. The && symbol chains the two commands together, ensuring that the upgrade only occurs after the update is complete.
Verify Docker Installation: Open your Ubuntu terminal and check if Docker is already installed:docker --version
If Docker is installed, you'll see the version number displayed. If not, proceed with the installation. Install Docker: Use the following commands to install Docker:sudo apt update && sudo apt install docker.io -y sudo systemctl enable --now docker
This sequence first updates the package list, then installs Docker (docker.io), using -y to automatically accept all prompts. Finally, it enables the Docker service so it starts automatically on boot and starts it immediately using systemctl enable --now docker.
Configure the NVIDIA Repository: Add the NVIDIA Container Toolkit repository to your system's package manager:curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \ && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \ sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
This command securely adds the required repository to your system, enabling the installation of the toolkit. It uses curl to download the necessary files and gpg to verify their authenticity, preventing malicious installations. sed is used to modify the repository URL, adding a signature verification component for enhanced security. Update the Package List: Refresh your system's package list to include the newly added repository:sudo apt-get update
Install the NVIDIA Container Toolkit: Install the toolkit:sudo apt-get install -y nvidia-container-toolkit
This command installs all necessary components of the NVIDIA Container Toolkit, enabling GPU support within Docker containers.
Run Ollama: Execute the following command in your Ubuntu terminal:docker run -d \ --gpus all \ #Grants access to all GPUs if available. Comment out if not using a GPU -v ollama:/root/.ollama \ # Mounts a local directory named 'ollama' to the Ollama data directory inside the container -p 11434:11434 \ #Maps port 11434 on the host to port 11434 in the container --security-opt=no-new-privileges \ # Enhances security by limiting the container's privileges --cap-drop=ALL \ #Drops all capabilities except those explicitly added --cap-add=SYS_NICE \ # Allows the container to adjust process priorities --memory=8g \ # Allocates 8GB of RAM to the container --memory-swap=8g \ # Allows 8GB of swap space for the container --cpus=4 \ # Allocates 4 CPU cores to the container --read-only \ #Mounts the container's filesystem as read-only for better security --name ollama \ # Assigns the name "ollama" to the container for easier management ollama/ollama
This command runs the ollama/ollama Docker image in detached mode (-d), allocating resources and setting security parameters. The -v flag creates a volume mount, persisting Ollama data even if the container is removed. The -p flag maps ports for external access. The flags --security-opt, --cap-drop, and --cap-add are crucial for enhancing security by limiting container privileges. The --memory, --memory-swap, and --cpus flags specify the resource limits for the container. Finally, --read-only and --name improve the security and manageability of the container.
Run DeepSeek-R1: Execute the following command within your Ubuntu terminal to start DeepSeek-R1 (replace 7b with the desired version if needed):docker exec -it ollama ollama run deepseek-r1:7b
This command executes the ollama run deepseek-r1:7b command inside the already running ollama container. -it assigns a pseudo-TTY and keeps stdin open, letting you interact with DeepSeek-R1.Interacting with DeepSeek-R1: Now you can interact with DeepSeek-R1. To exit the DeepSeek-R1 session, type /bye.
Start the Ollama Container: docker start ollama
Run DeepSeek-R1: docker exec -it ollama ollama run deepseek-r1:7b
0 comments:
Post a Comment