You've probably heard about ChatGPT, Claude, or Gemini. These are Large Language Models (LLMs), AI systems trained to understand and generate text. They're useful, but there's a tradeoff. Every time you use them, your information goes to someone else's servers. In my work, I sometimes deal with code, client material, or ideas that I do not want to send to an external AI. That made me curious about running models locally.

Then I discovered LM Studio. It's a simple program that lets you run AI models (like ChatGPT) on your own computer. Your conversations stay local, and you don't need any coding knowledge.

In this blog, I'll walk you through what I learned about LM Studio. I'll cover everything from downloading LM Studio to running your first model, what I discovered about the different options, and why this approach is useful when privacy matters.

Running local AI models used to feel like something only developers or people with powerful machines could do. That has changed. Tools like LM Studio make the setup much easier, and there are now many smaller and larger models you can run depending on what your computer can handle.

That matters because AI is becoming part of everyday work. People use it to write, summarize, brainstorm, review code, and organize information. But when everything goes through an online service, you also have to think about where your data is going.

With a local model, your conversations stay on your own machine. They are not sent to someone else's servers, which is useful if you work with sensitive information, client material, or code you want to keep private. It also gives you more freedom. After the setup, you can use the model locally without paying per message or needing a monthly subscription. You can use it once or a hundred times, it does not matter.

And because you run the model yourself, you also get more control. You can choose which model to use, how it responds, and when to use it. If you want it to be more creative, more direct, or better suited to a specific task, you can adjust it.

LM Studio is a program you can download for free that lets you run local AI models on your computer. You can chat with them directly within LM Studio, or if you're a programmer, start up a local server that your applications can talk to using HTTP requests.

It's designed to be straightforward. No need to understand code or complicated technical stuff. It works on Windows, Mac, and Linux.

1. Download and Install LM Studio

Go to lmstudio.ai and download the version for your computer. Just install it like you would install any other program.

2. Look at the Available Models

When you search for models in LM Studio, you'll see a list of different options you can choose from. These are models made by companies and individuals that have been shared publicly.

Keep in mind that different models have different requirements. The more RAM your computer has, the larger and more capable models you can run. Smaller models like Phi 2 work on 8GB machines. Larger models like Gemma 26B or Qwen need more memory to run smoothly. So, consider what your hardware can handle when choosing.

LM Studio will also tell you if your computer can run a model.

Some popular choices that work well:

• Mistral 7B: Good all-around model, fast, follows instructions well

• Llama 2 7B or 13B: Solid choice, widely used, lots of information online about it

• Phi 2: Smaller and faster, good if your computer isn't super powerful

• Gemma 2 9B or Gemma 26B: Great for regular conversations and answering questions. Gemma 26B is more powerful if you have a better computer.

• Qwen: Excellent if you're working with code or programming questions. Very good at understanding code.

• Neural Chat 7B: Made specifically to be good at conversations

Different models are better at different things. If you're mainly chatting and asking questions, go with Mistral or Gemma. If you're working with code, Qwen is a solid choice.

My suggestion: start with Mistral 7B. It's a great balance. Not huge, not tiny, and it works well.

3. Download your first large language model (LLM)

Search for "Mistral 7B" in LM Studio. You'll see several versions of the same model with names like Q4_K_M, Q5_K_M, Q8_0, or F32. These names basically tell you how compressed the model is. More compression means a smaller file and lower RAM usage, but slightly lower quality. Less compression means better quality, but it needs more memory and may run slower.

For a first install, the safest choice is usually Q4_K_M. It gives a good balance between quality, speed, and memory usage.

If your computer has 8-12 GB RAM, start with Q4_K_M. If you have 16 GB or more, you can try Q5_K_M or Q6_K for better quality. If LM Studio says the model is too large for your machine, pick a smaller version.

4. Load the model and start using it

Once the download is done, go to the "Chat" section in LM Studio. Pick your downloaded model from the list. LM Studio now loads the model into your computer's memory.

The first load takes some time. LM Studio is moving the whole model from your hard drive into your memory. This might take 30 seconds to a few minutes depending on your computer.

Once it's ready, you'll see a chat window. You can now chat with your local AI.

LM Studio can run as a local server on your computer so other programs can use your model. If you know how to program, you can build applications that make HTTP requests to your local model.

Here you'll find a simple example to ask your model a question from code: https://lmstudio.ai/docs/developer/rest/quickstart

In the LM Studio chat window, you can adjust how the model talks:

Temperature: Lower numbers (like 0.1 to 0.3) make the model more focused and consistent. Higher numbers (0.8 to 1.0) make it more creative and random.

Max Tokens: This controls how long the answer can be. See https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them for a detailed explanation about tokens and their usage.

System Prompt: This is the secret weapon. You tell the model how to behave. This is like telling a person what job they should do.

Example system prompt for a senior developer who makes random jokes:

[You are a senior software engineer who makes a lot of jokes. Answer technical questions clearly. Don't just answer but think before you speak.]

The system prompt totally changes how the model acts and what it focuses on.

I've used a local LLM several times already, mostly as a practical helper while working. It can help draft instructions, improve documentation, review code, and suggest things I might have missed.

I also like using it as a brainstorming partner. Because it runs locally, I can focus on the problem instead of thinking about where my data is going. That makes it useful when I'm working with client material, unfinished ideas, or code I do not want to send to an external service.

Another useful case is organizing information. You can paste in messy text and ask the model to pull out specific details or turn it into something more structured. It is also useful for testing ideas before using paid cloud services or building something more serious.

"The responses are super slow"

• Check if your computer is using its graphics card (if it has one). The settings in LM Studio can help with this.

• Try a smaller model. Some are lighter weight than others.

"The model keeps saying wrong things"

• Different models are better at different things. Try a different one.

• Change how you ask the question. Be more specific.

• Adjust the settings. Sometimes lower temperature helps with accuracy.

"I'm running out of space on my hard drive"

• LLMs take up a lot of space. You need about 5 to 15 gigabytes per model.

• Delete models you aren't using. LM Studio makes this easy.

• Use an external hard drive for extra space.

When I started with LM Studio, I was looking for a way to run models without sending everything to the cloud. What surprised me was how straightforward it is. I expected more friction and more setup hassle. But in the end, it was mostly download, pick a model, and start talking.

The interesting part has been seeing how different models behave. Gemma works well for straightforward questions. Qwen is useful when I'm working with code. Each model has its own strengths, and that flexibility matters when you're exploring what works for your specific needs.

Cloud services still have advantages. But having the option to run things locally changed how I think about this technology. It's no longer something distant that you only access through a service. It's something you can control.