Running AI Locally: What Your Laptop Can Handle

A laptop can run a language model without sending each prompt to a remote service. Whether the experience is useful depends on the model, available memory and the length of the work you give it. A successful download is only the first test.

Local AI is most attractive when you need offline access, predictable access to a particular model or tighter control over documents. It does not automatically make answers more accurate.

Start with available memory

Model files need storage space, but running a model also needs working memory. The operating system, browser and other applications already consume part of it. A computer advertised with a certain amount of RAM does not make all of that memory available to inference.

LM Studio’s system requirements recommend at least 16 GB of RAM for its supported desktop setups, while smaller models and shorter contexts can work on some 8 GB machines. Those are application guidelines, not a promise that every model will fit.

On a machine with a separate graphics card, GPU memory can be another constraint. Some systems split work between the GPU and CPU. On supported Apple silicon computers, unified memory changes that division, but the model still competes with other applications.

Check the application’s current support for your operating system and hardware before downloading a large model. Compatibility and comfortable performance are separate questions.

Can you run an LLM locally without a GPU?

Yes, but expect it to be slower. The open-source llama.cpp project supports ordinary x86 processors through AVX, AVX2, AVX512 and AMX instructions, treats Apple silicon as a first-class platform, and can split a model between CPU and GPU “to partially accelerate models larger than the total VRAM capacity.” It also supports 1.5-bit to 8-bit integer quantization to reduce memory use.

So the question is less “GPU or not” than “where do the weights sit.” A model held entirely in graphics memory (VRAM) runs fastest. A model that spills into system RAM still runs, more slowly. On Apple silicon, the GPU and CPU share unified memory.

How much VRAM does a local LLM need?

Start with the weights. Their size is roughly the parameter count times the bytes per parameter: about 2 bytes in BF16, 1 byte at 8 bits and a little over half a byte in common 4-bit formats. As a worked example, a 7-billion-parameter model needs about 14 GB in BF16, about 7 GB at 8 bits and about 4 GB at 4 bits for weights alone.

That is a floor, not the total. Runtime buffers add a margin, and the conversation itself needs a KV cache that grows with context length, explained in our KV cache memory guide. Our memory calculator applies the weights rule to every model in the open-weight model register, adds 10% for runtime buffers and shows whether a model fits on the GPU or only with offloading to RAM. It excludes the context cache, so leave headroom.

Model size is a trade-off

A model with more parameters often requires more memory. Quantization stores its numerical weights at reduced precision, making some models smaller and easier to run. It can also change answer quality. Two downloads bearing the same model name may therefore behave differently.

Choose by task rather than by the largest file your computer can load. A compact model may handle rewriting short text adequately while struggling to follow a complex document. A larger model that responds too slowly may be a poor fit for interactive work.

Record the exact model, quantization and application version when comparing results. Otherwise, an apparent improvement may come from a different runtime or setting rather than the model itself.

Long documents use more than disk space

The context window is the material available to the model during a request. Longer contexts consume additional resources. The maximum advertised context length is not a guarantee that your laptop can use it comfortably.

Document chat applications may retrieve selected passages instead of sending the whole file into every request. This can reduce the load, but it introduces another failure point: the relevant passage might not be retrieved.

Test a question whose answer is near the end of a document, a question that requires two separate sections and a question the document cannot answer. Ask for supporting passages and inspect them. A fluent response is not evidence that the entire file was read.

What offline actually means

According to LM Studio’s offline documentation, using downloaded models, processing documents and running its local server can work without an internet connection. Searching for models, downloading runtimes and checking for updates require connectivity.

That distinction should guide your test. Download what you need, disconnect from the network and repeat an ordinary task. If your workflow uses a web search tool or a remote connector, those parts will still need access.

Local processing also does not eliminate ordinary device risks. Documents may remain in chat history, operating-system backups or synced folders. A local server exposed to a network has a different access boundary from an application used only on the laptop.

Run a small trial before buying hardware

Use a representative document and a task you already know how to complete. Measure time to the first useful answer, total response time and the amount of correction required. Then repeat while your normal work applications are open.

Listen for sustained cooling noise and check whether the laptop remains comfortable to use. For battery work, compare a short real session with your normal workload. These observations are more useful than a single impressive demonstration.

Keep the cloud alternative in the comparison. Local processing can offer control and offline availability, while a hosted service may handle larger models without a hardware purchase. Our AI workflow cost guide explains how to compare the full job.