Setting Up a Free Local Large Language Model on Your PC

Published on 22th June 2025 by Simon Fearby

In this post you will learn how to install and run a free, private large language model (LLM) like Llama 3 on your Windows PC using Ollama. No cloud or API keys required./p>

Why am I setting up a local LLM (Ollama)?

I want to setup a local LLM running on my Windows 11, to automatically process (callable from C#) post content in order to create metadata for blog posts.

I am building a WP2Static C# Application to assist me import my old WordPress blog posts to static HTML.

I want to automate as much of this import possible as possible and having accurate and up-to-date Meta data in posts is important.

For me I am hoping to integrate this into a C# program I am writing to export WordPress to Static HTML.

Goals:

• Accepts your plain English content as input
• Returns metadata summaries, titles, or tag suggestions (as JSON)
• Can be called from C# code inside my converter/import tool
• Lets me avoid using external services like OpenAI or Hugging Face
• All for $0

What is Ollama?

Ollama is a free, open-source tool that lets you run Large Language Models (LLMs) locally on your own Windows, macOS, or Linux machine — no internet or cloud required after setup.

What Does Ollama Do?

• Downloads pre-built LLM models (like LLaMA 3, Mistral, Gemma, etc)
• Runs them efficiently on your CPU or GPU
• Exposes a local API (http://localhost:11434) that you can call from any program (like your C# app)
• Handles:
• Tokenization
• Model loading and caching
• Text generation
• Streaming responses (optional)

Models you can run with Ollama locally

• llama3 – Meta’s LLaMA 3 model (good for summaries, general-purpose text)
• mistral – Small, fast open-weight model
• codellama – Code-specialized LLM
• gemma – Google’s model family
• llava – Multimodal vision + language model

Best use of each Model

Why Use Ollama Locally?

• Privacy: All content stays on your machine.
• No tokens or API keys: No OpenAI/Gemini charges or rate limits.
• Performance: Often faster than calling cloud APIs (especially for batch processing).
• Offline use: Works without an internet connection once models are downloaded.

How to set up Ollama

• Go to https://ollama.com/download
• Download and run the Windows installer (OllamaSetup.exe)
• Once installed, open PowerShell and run: ollama run llama3

1. Download OllamaSetup.exe

2. Run the Installer

3. Installing Files

4. I Opened a Windows Command Prompt and checked to see if ollama was working (it was)

Microsoft Windows [Version 10.0.26100.4351]
(c) Microsoft Corporation. All rights reserved.

C:\Users\simon>ollama
Usage:
  ollama [flags]
  ollama [command]

Available Commands:
  serve       Start ollama
  create      Create a model from a Model file
  show        Show information for a model
  run         Run a model
  stop        Stop a running model
  pull        Pull a model from a registry
  push        Push a model to a registry
  list        List models
  ps          List running models
  cp          Copy a model
  rm          Remove a model
  help        Help about any command

Flags:
  -h, --help      help for ollama
  -v, --version   Show version information

Use "ollama [command] --help" for more information about a command.

4. Then I ran the command ollama run llama3 to download the

Microsoft Windows [Version 10.0.26100.4351]
(c) Microsoft Corporation. All rights reserved.

C:\Users\simon>ollama run llama3
...

5. Models are downloaded automatically

6. It took about an hour for all models to download on my PC

Testing the ollama llama3 model

llama3 Test 1:

I ran this command to test the llama3 ("ollama run llama3" was already running in my command prompt).

Summarize this article: "Electric cars use electricity stored in batteries. They are charged by plugging in and have no tailpipe emissions..."

Result:

The article is stating that electric cars (EVs) operate on a unique principle:

  1. They use electricity stored in rechargeable batteries.
  2. To replenish these batteries, EVs need to be plugged into an electrical power source.
  3. Unlike traditional gasoline-powered vehicles, EVs do not have a tailpipe and therefore produce zero emissions,
  making them an environmentally friendly option.

llama3 Test 2:

I then asked it to summarise this post (I copied and pasted the text): Migrating my website from upcloud to Linode Akamai for a Disaster Recovery Test

Result:

The article describes the author's experience with migrating their website from UpCloud to Linode Akamai for a disaster recovery test. The author created a Linode account, set up a Linux server, and transferred their domain registration to Linode.

The process involved:

1. Creating a Linode account and setting up a profile.
2. Generating an SSH key and adding it to the profile.
3. Registering a temporary domain via Porkbun.
4. Setting up a new server on Linode with a Linux distribution, firewall, and backups.
5. Configuring the server with Cloudflare, NGINX, PHP, MariaDB, and other necessary software.
6. Transferring their old website from UpCloud to the new Linode server.

The author notes that this was a successful disaster recovery test and they were able to set up a new server on a different host in under 1 hour. The
article provides step-by-step instructions and screenshots for each part of the process, making it useful for those looking to migrate their website
from one cloud provider to another or set up a disaster recovery plan.

llama3 Test 3:

I then asked it to summarise this Zen Koan: Is That So?

  The Zen master Hakuin was praised by his neighbors as one living a pure life.

  A beautiful Japanese girl whose parents owned a food store lived near him. Suddenly, without any warning, her parents discovered she was with child.

  This made her parents angry. She would not confess who the man was, but after much harassment at last named Hakuin.

  In great anger the parent went to the master. "Is that so?" was all he would say.

  After the child was born it was brought to Hakuin. By this time he had lost his reputation, which did not trouble him, but he took very good care of the child. He obtained milk from his neighbors and everything else he needed.

  A year later the girl-mother could stand it no longer. She told her parents the truth – the real father of the child was a young man who worked in the fishmarket.

  The mother and father of the girl at once went to Hakuin to ask forgiveness, to apologize at length, and to get the child back.

  Hakuin was willing. In yielding the child, all he said was: "Is that so?"

Result:

The article tells a story about Zen master Hakuin, who is known for living a pure life. A young woman from a nearby food store becomes pregnant and refuses to reveal the identity of the father. Her parents eventually discover that she is with child and accuse Hakuin of being the father. Despite losing his reputation, Hakuin takes care of the child until it is born.

When the truth comes out that the real father is actually a young man who works in the fish market, the girl's parents go to Hakuin to apologize and ask for the child back. However, instead of being angry or vindictive, Hakuin simply says "Is that so?" and hands over the child without hesitation.

Testing the ollama codellama model

Codellama Test 1:

I exited the llama3 model and ran the command "ollama run codellama:instruct" then entered "Write some C# code to connect to a local MySQL database and query a database."

Result #1:

C:\Users\simon>ollama run codellama:instruct
pulling manifest
pulling 3a43f93b78ec: 100% ▕██████████████████████████████████████████████████████████▏ 3.8 GB
pulling 8c17c2ebb0ea: 100% ▕██████████████████████████████████████████████████████████▏ 7.0 KB
pulling 590d74a5569b: 100% ▕██████████████████████████████████████████████████████████▏ 4.8 KB
pulling 2e0493f67d0c: 100% ▕██████████████████████████████████████████████████████████▏   59 B
pulling 7f6a57943a88: 100% ▕██████████████████████████████████████████████████████████▏  120 B
pulling 316526ac7323: 100% ▕██████████████████████████████████████████████████████████▏  529 B
verifying sha256 digest
writing manifest
success
>>> Write some C# code to connect to a local MySQL database and query a database.

Here is an example of how you can connect to a local MySQL database in C# and query the database:
```
using System;
using MySql.Data.MySqlClient;

namespace ConsoleApp
{
    class Program
    {
        static void Main(string[] args)
        {
            // Connect to the database
            string connectionString = "server=localhost;user
id=your_username;password=your_password;database=your_database";
            MySqlConnection conn = new MySqlConnection(connectionString);
            conn.Open();

            // Query the database
            string query = "SELECT * FROM your_table WHERE column1 > 5";
            MySqlCommand cmd = new MySqlCommand(query, conn);
            MySqlDataReader reader = cmd.ExecuteReader();

            // Read the data from the database
            while (reader.Read())
            {
                Console.WriteLine("Column1: " + reader["column1"].ToString());
                Console.WriteLine("Column2: " + reader["column2"].ToString());
                Console.WriteLine("---------------------------");
            }

            // Close the connection
            conn.Close();
        }
    }
}
```
This code uses the `MySqlConnection` class to connect to a local MySQL database, and the `MySqlCommand` class to
execute a query on the database. The `MySqlDataReader` class is used to read the data from the query results.

In this example, the connection string specifies the server as "localhost", the user name and password for the
connection, and the name of the database to connect to. The query selects all rows from a table named "your_table"
where the value in column1 is greater than 5.

Note that you will need to replace "your_username", "your_password", "your_database", and "your_table" with the
appropriate values for your own database and table.

Testing the ollama gemma model

Gemma Test 1:

I exited the codellama:instruct model and ran the command "ollama run gemma" then entered "Write a short intro paragraph for a blog post titled: “Setting Up a Free Local Large Language Model on Your PC"

Result:

Setting Up a Free Local Large Language Model on Your PC
Ever dreamt of having your own personal AI assistant that could understand natural language and generate
human-like responses? Well, that dream can become reality with the rise of large language models (LLMs). While
these powerful AI models have traditionally been available only through cloud services, recent advancements allow
you to run them directly on your own PC for free. In this blog post, we'll walk you through the process of setting
up a free local large language model on your PC, empowering you to harness the incredible potential of AI without
any subscription fees.
        

Gemma Test 2:

I entered "Write a short intro paragraph for a blog post titled: “The biggest AI inventions in the last 20 years"

Result:

The past two decades have witnessed groundbreaking advancements in the field of artificial intelligence. From
language translation to image recognition and machine translation, AI has revolutionized countless industries and
aspects of human life. This rapid progress has resulted in numerous remarkable inventions that have shaped our
world. In this blog post, we explore the most significant AI inventions of the past 20 years, highlighting their
impact on society and speculating on their potential for future innovation. Prepare to delve into the fascinating
world of AI and discover the transformative power of these groundbreaking technologies.
        

Gemma Test 3:

I entered "Write about the last 100 years of space exploration"

Result:

## A Journey to the Stars: 100 Years of Space Exploration

The human desire to explore the cosmos has captivated minds for centuries. However, it wasn't until the dawn of
the 20th century that humanity took its first tentative steps towards the stars. From the pioneering missions of
the Mercury and Gemini programs to the thrilling discoveries of the Hubble telescope and the Mars rovers, the past
century has been an extraordinary journey of exploration and innovation.

**The Early Explorers:**

The history of space exploration begins with the launch of Sputnik by the Soviet Union in 1957. This pivotal event
sparked the "Space Race," leading to the United States' triumph with the Apollo missions and the subsequent moon
landings. Early missions focused on achieving technological milestones, capturing stunning images of our celestial
neighbors and pushing the boundaries of human knowledge.

**Beyond the Moon:**

As technology advanced, space exploration ventured beyond our natural satellite. Missions to Mars, Jupiter, and
beyond revealed previously unimaginable landscapes and scientific wonders. Hubble telescope opened up a new window
into the cosmos, revealing the breathtaking beauty and vastness of the universe.

**The Future Frontier:**

Present-day space exploration is characterized by ambitious projects like the Mars Sample Return mission and the
Artemis program, which aims to establish a sustainable human presence on the Moon. With renewed interest in space
travel and private companies like SpaceX driving innovation, the future promises further groundbreaking
discoveries and scientific breakthroughs.

**Conclusion:**

The last century has been a remarkable era of space exploration, filled with thrilling triumphs and inspiring
discoveries. As technology continues to evolve, we can only imagine the exciting possibilities that await us in
the decades to come.
        

Gemma Test 4:

I entered "Write about the potential of mining asteroids?"

Result:

## The Potential of Mining Asteroids

As our thirst for resources grows and the demand for precious metals intensifies, astronomers and engineers have
their sights set on a potential goldmine: asteroids. These rocky space objects, remnants of the early solar
system, hold an abundance of valuable materials like iron, nickel, copper, and even gold.

**The Challenges:**

Mining asteroids poses significant challenges. These objects are often irregularly shaped, making extraction
difficult. Additionally, the harsh environments, extreme temperatures, and the sheer distance from Earth require
advanced technology and unwavering ingenuity.

**Methods of Extraction:**

Scientists envision various methods for asteroid mining. One approach involves using robotic arms to collect
material from the surface. Another technique utilizes lasers to vaporize the asteroid's surface, extracting
valuable metals through the vapor.

**Economic and Environmental Impact:**

Asteroid mining could potentially address the resource scarcity concerns of the future. The economic benefits are
undeniable, but environmental concerns also arise. The extraction process could pollute space, raising questions
about the responsibility and sustainability of such activities.

**Future Prospects:**

While asteroid mining remains in its nascent stages, technological advancements and commercial interests are
driving its potential. With collaborative efforts between governments, private companies, and researchers, this
ambitious endeavor could become a reality in the not-so-distant future.

**Conclusion:**

Asteroid mining offers a promising solution to our resource challenges, but only through responsible and
sustainable practices can we harness the benefits of this groundbreaking technology without compromising the
integrity of our celestial neighborhood.
        

Connecting to Ollama from a C# Winforms App (Proof on Concept Test App)

I wanted to use the llamam3 model to do the following..

• Summarize this blog post in 2 sentences. Then list 5 relevant tags.
• Then choose 1 category from this list: "Servers", "Electric Vehicle", "Technology", "Software", "Development".
• If the post does not clearly fit any of these, use "Misc". "
• Estimate read_time in the format "X minutes" based on the word count.
• Respond in JSON format like this: { \"description\": \"...\", \"tags\": [\"...\"], \"category\": \"...\", \"read_time\": \"...\", \"word_count\": 123 }\n\n";

This can be done with this prompt (stored in a string called "prompt")

string prompt =
"Summarize this blog post in 2 sentences. Then list 5 relevant tags. " +
"Then choose 1 category from this list: [\"Servers\", \"Electric Vehicle\", \"Technology\", \"Software\", \"Development\"]. " +
"If the post does not clearly fit any of these, use \"Misc\". " +
"Estimate read_time in the format \"X minutes\" based on the word count. " +
"Respond in JSON format like this: { \"description\": \"...\", \"tags\": [\"...\"], \"category\": \"...\", \"read_time\": \"...\", \"word_count\": 123 }\n\n";

I prepared the contents of the body of this post (Efergy E-Max Wireless Energy Monitor from Reduction Revolution) by removing html tags and unwanted text and characters. This was stored in a string called "sHTMLBodyText")

I checked to see if the prompt was over 12000 charachters etc

I appended the html to the prompt

//Add the HTML Text to the Prompt
prompt = prompt + sHTMLBodyText;

I prepared the POST JSON to send to the API Endpoint

var payload = new Dictionary
{
    { "model", "llama3" },
    { "prompt", prompt },
    { "stream", false }
};

I posted this to my local endpoint

string jsonPayload = new JavaScriptSerializer().Serialize(payload);
var request = (HttpWebRequest)WebRequest.Create("http://localhost:11434/api/generate");
request.Method = "POST";
request.ContentType = "application/json";

byte[] byteArray = Encoding.UTF8.GetBytes(jsonPayload);
request.ContentLength = byteArray.Length;

using (Stream dataStream = request.GetRequestStream())
    dataStream.Write(byteArray, 0, byteArray.Length);

using (var response = (HttpWebResponse)request.GetResponse())

I processed the payload

using (var reader = new StreamReader(response.GetResponseStream()))
{
    labelAIStatus.Text = "AI Status: Reading AI response";
    Application.DoEvents();
    System.Threading.Thread.Sleep(500);

    string rawJson = reader.ReadToEnd();

    var serializer = new JavaScriptSerializer();
    var wrapper = serializer.Deserialize>(rawJson);
    string aiWrappedResponse = wrapper["response"].ToString();

    int start = aiWrappedResponse.IndexOf('{');
    int end = aiWrappedResponse.LastIndexOf('}');

    if (start >= 0 && end > start)
    {
        string cleanJson = aiWrappedResponse.Substring(start, end - start + 1);
        var aiData = serializer.Deserialize>(cleanJson);

        // Description
        string desc = aiData["description"].ToString();
        labelAIStatus.Text = "AI Status: Processing \"description\"";
        Application.DoEvents();
        System.Threading.Thread.Sleep(200);

        // Category
        labelAIStatus.Text = "AI Status: Processing \"category\"";
        Application.DoEvents();
        System.Threading.Thread.Sleep(200);


        string category = aiData["category"].ToString();
        
        // Read Time
        string readTime = aiData["read_time"].ToString();
        labelAIStatus.Text = "AI Status: Processing \"read_time\"";
        Application.DoEvents();
        System.Threading.Thread.Sleep(200);

        // Word Count
        int wordCount = Convert.ToInt32(aiData["word_count"]);
        labelAIStatus.Text = "AI Status: Processing \"word_count\"";
        Application.DoEvents();
        System.Threading.Thread.Sleep(200);

        // Tags
        labelAIStatus.Text = "AI Status: Processing \"tags\"";
        Application.DoEvents();
        System.Threading.Thread.Sleep(200);

        List tagList = new List();
        if (aiData["tags"] is System.Collections.ArrayList rawTagList)
        {
            
            foreach (var tag in rawTagList)
            {
                string camelTag = ToCamelCase(tag.ToString());
                tagList.Add("\"" + camelTag + "\"");
            }

        }
        string tagListCsv = string.Join(", ", tagList);

        // Replace placeholders
        textPostJSON.Text = textPostJSON.Text.Replace("TODO_PAGE_DESCRIPTION", desc);
        //textPostJSON.Text = textPostJSON.Text.Replace("TODO_JSON_CATEGORY_ARRAY_HERE", category);
        textPostJSON.Text = textPostJSON.Text.Replace("TODO_JSON_CATEGORY_STRING_HERE", $"\"{category}\"");
        textPostJSON.Text = textPostJSON.Text.Replace("TODO_JSON_TAGS_ARRAY_HERE", tagListCsv);
        textPostJSON.Text = textPostJSON.Text.Replace("TODO_PAGE_READ_TIME_HERE", readTime);
        textPostJSON.Text = textPostJSON.Text.Replace("TODO_PAGE_BODY_WORD_COUNT_HERE", wordCount.ToString());
    }
    else
    {
        labelAIStatus.Text = "AI Status: No valid JSON found in response.";
        Application.DoEvents();
        System.Threading.Thread.Sleep(200);
    }
}

Conclusion

I was able to get llama3 AI locally to read my blog post HTML (Efergy E-Max Wireless Energy Monitor from Reduction Revolution), generate a description, category and tags, Estimate the read time and word count and output as JSON

{
  "description": "The author reviews the Energy monitor from Reduction Revolution, comparing it to their previous experience with Power Pal Energy Monitor. They discuss its features, pros, and cons.",
  "category": ["Servers"],
  "tags": ["Energy Monitor", "Power Pal", "Renewable Energy", "Electric Vehicle", "Smart Meter"],
  "read_time": "5 minutes",
  "word_count": 466, 
}

This will open up a lot of automation for me from C#.

Thanks for Reading

v1.0