Building NeoGPT: My First Self-Hosted AI Assistant

1. Goal and hardware

This project started with a simple goal: build a completely private AI assistant that runs entirely on my own hardware instead of relying on cloud AI providers. I wanted to understand how modern LLMs actually work, how they're hosted, and how I could customize one for my own needs.

The server is an older laptop, repurposed as a dedicated AI host:

Intel Core i5-10210U
16 GB RAM
Ubuntu Server
Docker for containerized applications

Design intent: the goal wasn't the fastest AI server — it was a practical learning environment I can keep expanding.

2. Base OS and Docker

The first step was installing Ubuntu Server and configuring the machine as a dedicated AI host. After installation I:

Updated the operating system.
Enabled SSH for remote management.
Configured a static local IP.
Verified network connectivity.

Instead of installing applications directly onto the OS, I chose Docker. Every service runs in its own isolated container, which makes deployments cleaner and much easier to update and back up. Everything in the NeoGPT stack runs inside Docker containers.

# Update, then verify Docker is running
sudo apt update && sudo apt full-upgrade -y
docker --version
docker ps

3. Local inference with Ollama

The core of the stack is Ollama, the local inference engine that serves Large Language Models entirely on my own hardware. Unlike cloud AI services, every prompt stays on my network — no conversations are sent to OpenAI, Anthropic, Google, or any external provider.

# Run Ollama as a container, persisting models in a named volume
docker run -d --name ollama \
  -v ollama:/root/.ollama \
  -p 11434:11434 \
  ollama/ollama

Once Ollama was running, I began downloading and comparing models:

docker exec -it ollama ollama pull llama3
docker exec -it ollama ollama pull qwen2.5:3b
docker exec -it ollama ollama list

Lesson learned: larger models are significantly slower on a CPU-only laptop. Running several locally made it obvious how much hardware — especially a GPU — matters for local AI.

4. Open WebUI (the interface)

To avoid interacting with Ollama only through the terminal, I installed Open WebUI — a modern, ChatGPT-style interface that still keeps everything local. It connects directly to Ollama and lets me chat with any installed model from a browser. This became the primary interface for NeoGPT.

docker run -d --name open-webui \
  -p 3000:8080 \
  -e OLLAMA_BASE_URL=http://<server-ip>:11434 \
  -v open-webui:/app/backend/data \
  ghcr.io/open-webui/open-webui:main

Connecting the containers

The first real troubleshooting step was verifying communication between the two containers. I checked the Docker environment variables and confirmed the Ollama API endpoint (OLLAMA_BASE_URL) was pointed at the right host and port. Once the containers could reach each other, Open WebUI immediately detected the installed models.

5. Keeping it internal-only (DNS + reverse proxy)

A major design goal was privacy. Rather than exposing NeoGPT to the internet, I wanted it reachable only inside my home network, so every conversation, prompt, and model stays local.

Local DNS with Pi-hole

Initially Open WebUI was only reachable at http://neogpt.home:3000, and I wanted something cleaner. My Pi-hole already handles DNS for the network, so I added a local DNS record mapping the hostname to the server's internal IP. This is also where the DNS-vs-reverse-proxy distinction finally clicked for me.

DNS vs. reverse proxy: at first I assumed a reverse proxy somehow replaced DNS. They actually solve different problems. DNS answers "what IP does this hostname belong to?" A reverse proxy answers "which web application should receive this request?" Once I saw that separation, the whole architecture made sense.

Nginx Proxy Manager

To drop the :3000 from the URL, I installed Nginx Proxy Manager and created a proxy host so requests to http://neogpt.home forward internally to the Open WebUI container on port 3000 — without exposing the application directly. I also learned that a reverse proxy can only proxy services it can actually reach on the network, whether local or on another reachable host.

The end result is a clean, internal-only URL backed by two cooperating pieces:

Pi-hole resolves neogpt.home to the server's internal IP.
Nginx Proxy Manager forwards that request to the right container and port.

6. Managing it all with Portainer

To make Docker easier to manage, I added Portainer — a graphical dashboard for the whole stack instead of relying only on the command line. From it I can:

Start and stop containers.
View logs.
Monitor resource usage.
Deploy new stacks.
Manage Docker volumes and networks.

This will make expanding NeoGPT much easier going forward.

7. What I'd do next

Build a fully custom web interface instead of relying on Open WebUI.
Create a personal knowledge database using Retrieval-Augmented Generation (RAG).
Give NeoGPT internet search capabilities.
Connect local APIs and automation tools.
Add long-term memory.
Integrate it with the rest of my homelab.
Create specialized assistants for cybersecurity, development, and home automation.

8. What I learned

This project taught me far more than simply running an AI model. I came away with a much better understanding of:

Docker containers and container networking.
Local AI inference and how LLMs are hosted.
Reverse proxies and how they differ from DNS.
Local DNS and internal web hosting.
Self-hosted infrastructure end to end.

Most importantly, I now have a fully private AI assistant running on hardware I control, with a solid foundation I can keep improving as I learn more.