Every time you paste something into a chatbot, it leaves your computer. For a grocery list, that's fine. For a spreadsheet of student grades, a client's unreleased brand strategy or a draft you're not ready to share, it's worth asking a simple question: does this really need to go to someone else's server?

For a lot of that work, the answer is no. You can run a language model on your own laptop, with no internet connection, no account and no monthly fee. It's less mysterious than it sounds, and it's a lot less work than it used to be.

This guide is about when that makes sense, what you need, and how to get started without wasting an afternoon downloading something your computer can't run.

Should you bother?

Let's be honest up front: cloud tools like ChatGPT, Claude and Gemini are easier to use and, for many tasks, more capable than anything you'll run at home. They have free tiers, they work on any device, and they get better without you doing anything.

Local AI earns its place in a few specific situations:

• You handle sensitive information. Student names, grades, feedback, health notes, contracts, client work under NDA. With a local model, nothing leaves your machine.
• You work without reliable internet. Classrooms with bad wifi, travel, or places where the connection comes and goes.
• You don't want another subscription. Local tools are free to run once you own the hardware. No usage caps, no "you've reached your limit, come back in three hours".
• You want control. The model doesn't change overnight. The app doesn't quietly start training on your data. You pick the version and keep it.

My own bias is toward tools I own and open source software, so I lean local whenever the task allows. But the right answer depends on the job. Plenty of people will use a cloud tool for brainstorming and a local model for anything with real names in it. That's a perfectly sensible split.

What a local model actually is

A language model is a very large file. It's the result of training on huge amounts of text, compressed into billions of numbers (called parameters) that capture patterns in language. When you "download a model", you're downloading that file. When you "run" it, an app loads it into memory and uses it to predict text, one piece at a time.

That's why the two numbers that matter most are the model's size and your computer's memory. If the file doesn't fit comfortably in memory, it either won't load or it will crawl.

The apps: LM Studio and Ollama

You need an app to load and talk to the model. Two are well established and work on Mac, Windows and Linux.

LM Studio is the friendliest place to start. It's a regular desktop app with a chat window, a built-in search for models, and a clear indicator of whether a given model is likely to fit on your machine. No terminal required. It's free, including for use at work, though the app itself is not open source.

Ollama is open source and very lightweight. It started as a command-line tool (you type ollama run plus a model name and you're chatting) and now also has a simple desktop app on Mac and Windows. It runs quietly in the background, which makes it popular for connecting other apps to a local model.

If you've never done this before, start with LM Studio. If you like the terminal, or you want other tools to talk to your model, try Ollama. You can have both installed; they don't fight.

Picking a model your computer can run

Models are described by their parameter count: 3B, 7B, 14B, 32B, 70B and so on (B for billion). More parameters usually means better reasoning and writing, and always means more memory.

Model names and versions change every few months, so don't get attached to a specific one. The sizes, though, are a stable guide. Here's a rough rule of thumb for models compressed to 4-bit (more on that below):

• 3B to 4B: Approximate file size: 2 to 3 GB; Comfortable with: 8 GB of memory
• 7B to 9B: Approximate file size: 4 to 6 GB; Comfortable with: 8 to 16 GB
• 12B to 14B: Approximate file size: 8 to 9 GB; Comfortable with: 16 GB
• 27B to 32B: Approximate file size: 16 to 20 GB; Comfortable with: 32 GB
• 70B: Approximate file size: around 40 GB; Comfortable with: 64 GB or more

On a Mac with Apple Silicon, "memory" is the unified memory listed in About This Mac. On a Windows or Linux PC with a dedicated graphics card, the card's video memory (VRAM) matters most, and regular RAM picks up what doesn't fit, more slowly.

For everyday teaching and design tasks (drafting a lesson plan, rewriting instructions, summarizing a reading, generating rubric language) a model in the 7B to 14B range is usually enough. Bigger models help with long, nuanced documents and trickier reasoning. They don't make a short email better.

Leave headroom. Your browser, Figma and everything else also need memory. A model that technically fits but leaves nothing for the rest of your system will make the whole computer miserable.

Quantization, in one paragraph

Quantization is how models get small enough to run at home. The original model stores each number with high precision (16 bits). A quantized version stores them with fewer bits: 8, 5, 4, or even less. Think of it like exporting a JPG at a lower quality setting: the file shrinks a lot and, up to a point, you barely notice. A 4-bit version is roughly a quarter to a third of the original size. In model listings you'll see labels like Q8, Q5 or Q4 (often followed by letters like _K_M). Q4 or Q5 is the usual sweet spot. Go below Q4 only if you have to, since quality drops quickly from there.

Your first session, step by step

1. Check your memory. Write down how much RAM (or unified memory) and, if you have one, how much VRAM your machine has. This decides everything else.
2. Install LM Studio from its official site, lmstudio.ai.
3. Search for a model in the app's discover tab. Pick something in the size range from the table above, in a Q4 or Q5 version. LM Studio will warn you if it looks too big.
4. Download it. Several gigabytes, so do it on a good connection. This is the only step that needs internet.
5. Load the model and test it with a real task, not "hello". For example: "Write three learning objectives for a 90-minute introduction to color theory for first-year design students."
6. Adjust temperature if needed. Temperature controls how adventurous the output is. Around 0.7 is a reasonable default for writing; lower it (0.2 to 0.4) when you want consistent, factual answers like rubric criteria.

Then turn off your wifi and try it again. That moment is when local AI clicks for most people.

What about images?

You can generate images locally too, but the bar is higher. Image models want a good graphics card or an Apple Silicon Mac with plenty of memory, and the tools are less polished than LM Studio.

The most actively developed option is ComfyUI, which uses a node-based interface. If you've used node editors in 3D or compositing software, it will feel familiar. If you haven't, expect a learning curve. Simpler front ends exist, but they come and go faster than text tools do, so check what's currently maintained before investing time.

My honest advice: get comfortable with local text first. It's where the payoff is fastest.

Looking after your setup

A few habits save headaches later:

• Delete models you don't use. They're big. Both apps show you where models live and let you remove them.
• Update deliberately. New models appear constantly and many are genuinely better. But if one works well for a recurring task (say, turning your notes into student-friendly instructions), keep it until something clearly beats it.
• Back up your favorites to an external drive if your internet is slow. Downloading 20 GB twice is no fun.
• Keep a short log. Which model, which quantization, what you used it for, and how it did. After a month you'll know exactly which one to reach for.

A good first exercise

Take one task you do every week that involves information you wouldn't want to paste into a website. Maybe it's turning rough grading notes into clear feedback, or summarizing a confidential brief. Do it once with a local model. Note how long it took, what you had to fix, and whether the result was good enough.

If it was, you've just moved one piece of sensitive work back onto your own machine. If it wasn't, try the next size up, or accept that this particular task belongs in the cloud. Either way, you'll be deciding from experience rather than hype.