Self-Hosting Deepseek R1 with Ollama and LLM Studio: A Step-by-Step Guide

Clive Lancaster

Clive Lancaster

— 3 min read

Deep learning models like the Deepseek R1 series have revolutionized natural language processing, offering powerful capabilities for text generation, summarization, and more. If you're interested in self-hosting one of these models, such as the Deepseek R1 7B variant, using tools like Ollama and LLM Studio, this guide will walk you through the process.

By the end of this tutorial, you'll have a fully functional Deepseek R1 model running on your local machine or server, ready to handle custom tasks.

Table of Contents

  1. Introduction
  2. Prerequisites
  3. Step 1: Setting Up Ollama
  4. Step 2: Downloading and Installing Deepseek R1
  5. Step 3: Using LLM Studio for Fine-Tuning (Optional)
  6. Step 4: Running Inference
  7. Conclusion

Introduction

Self-hosting large language models (LLMs) can be resource-intensive but offers significant advantages, including privacy, customization, and reduced reliance on third-party APIs. Tools like Ollama simplify the deployment process, while platforms like LLM Studio enable advanced fine-tuning and experimentation.

In this guide, we'll focus on deploying the Deepseek R1 7B model, which is part of the Deepseek family of open-source LLMs. This model is known for its high performance and efficiency, making it an excellent choice for various applications.

Prerequisites

Before proceeding, ensure you meet the following requirements:

  1. Hardware Requirements:
    • A GPU with at least 8GB VRAM (e.g., NVIDIA RTX 2060 or higher).
    • At least 16GB of RAM.
    • Sufficient storage space (approximately 20GB for the model).
  2. Software Requirements:
    • Linux or macOS operating system.
    • Docker installed on your machine.
    • Python (optional, for advanced use cases).
  3. Basic Knowledge:
    • Familiarity with command-line interfaces.
    • Basic understanding of containerized environments.

Step 1: Setting Up Ollama

Ollama is a lightweight tool designed to make it easy to download, manage, and run large language models locally. Follow these steps to install and configure Ollama:

Installation

  1. Install Ollama:
  2. Verify Installation:

Run the following command to confirm that Ollama is installed correctly:

ollama version

For macOS:

brew install ollama

For Linux:

curl https://ollama.ai/install.sh | sh

Step 2: Downloading and Installing Deepseek R1

Now that Ollama is set up, let's download and install the Deepseek R1 7B model.

Download the Model

Verify the model installation:

ollama list

You should see deepseek/r1-7b listed among the available models.

Use the ollama pull command to download the Deepseek R1 7B model:

ollama pull deepseek/r1-7b
Note: This step may take some time depending on your internet connection and hardware specifications.

Step 3: Using LLM Studio for Fine-Tuning (Optional)

If you want to fine-tune the Deepseek R1 model for specific tasks or datasets, consider using LLM Studio. LLM Studio provides a user-friendly interface for training, evaluating, and deploying LLMs.

Installation

  1. Download LLM Studio:
    Visit the LLM Studio website and download the appropriate version for your operating system.
  2. Launch LLM Studio:
    Extract the downloaded file and run the application.
    • Import your dataset into LLM Studio.
    • Select the Deepseek R1 7B model as the base model.
    • Configure hyperparameters and start the fine-tuning process.

Fine-Tune the Model:

Tip: Refer to the LLM Studio documentation for detailed instructions on dataset preparation and fine-tuning workflows.

Step 4: Running Inference

Once the model is installed (and optionally fine-tuned), you can start generating text using Ollama.

Generate Text

Enter a prompt when prompted, and the model will generate a response based on your input.Example:

Prompt: Write a short story about a robot who discovers emotions.
Response: Once upon a time, in a futuristic city...

Open a terminal and run the following command to interact with the model:

ollama run deepseek/r1-7b

Use the Model Programmatically

For programmatic access, you can use Ollama's REST API. Here's an example using curl:

curl -X POST http://localhost:11434/api/generate \
     -d '{"model":"deepseek/r1-7b", "prompt":"What is the meaning of life?", "max_tokens":100}'

This will return JSON output containing the generated text.

Conclusion

Congratulations! You've successfully self-hosted the Deepseek R1 7B model using Ollama and explored optional fine-tuning with LLM Studio. With this setup, you can now leverage the power of Deepseek R1 for a wide range of applications, from chatbots to content generation.

While self-hosting LLMs requires some technical expertise and resources, tools like Ollama and LLM Studio significantly lower the barrier to entry. Experiment with different prompts, fine-tune the model for your specific needs, and unlock the full potential of Deepseek R1!

If you encounter any issues during the process, refer to the official documentation for Ollama and LLM Studio. Happy modeling!

Read more