How to run your own AI on a laptop
Updated:
Install a model runner such as Ollama, download an open model sized for your memory (a 7 to 8 billion parameter model fits in 16 GB of RAM) and start chatting. It works offline, and your data never leaves the computer.
How much memory you need
A rough rule: a compressed (quantized) model needs a bit more than half a gigabyte of memory per billion parameters. 8 GB is enough for small 3B models, 16 GB for 7 to 8B, 32 GB and more for 14B and up.
A graphics card makes answers several times faster, but it is not required to start.
Which model to pick
Pick by task, not by leaderboard. For chat and writing, a general instruct model. For code, a code model. For other languages, check that the model was trained on them: some small models write Russian or Kazakh poorly.
What a local model is good at
It is private, free after setup and works without internet. It is weaker than the best cloud models in open conversation, but it shines when connected to your own documents, because it answers from exactly your files.