What a million real questions say about small local models

Stanford researchers ran about a million real questions through local models, and the best local model for each question answered 88.7% of them. That backs the first belief on our about page (opens in a new tab), with two limits: the study covers single-turn questions, and its best single result came from a model far larger than the ones Base Drive will carry.

That belief reads: "Local, open, small models are good enough." We mean good enough "to do just about everything people actually use frontier models for."

The same entry is candid about where the belief came from: "Some researchers came to this conclusion by measuring a million real queries. We came to it based off vibes. Both hold up."

A million real questions

The research is Intelligence per Watt (opens in a new tab), a preprint from Stanford researchers posted on arXiv (2511.07885). The first version went up on November 11, 2025, and the latest revision (v7) on October 1, 2026.

The team ran more than 20 local models on 8 accelerators against about one million real single-turn chat and reasoning queries: 500,000 chat queries from WildChat, 500,000 reasoning questions from NaturalReasoning, and two smaller exam-style sets.

The paper counts success two ways. For reasoning and knowledge questions, the local answer had to match the benchmark's correct answer. For open-ended chat, the authors compared the local reply with one from Qwen3-235B, which the paper calls the top open model on LMArena as of August 2025. A judge model then ruled whether the local reply won or tied.

The 88.7% assumes each query goes to whichever local model handles it best, and a single model covers less, as the table shows. Intelligence per watt is the paper's measure of task accuracy per unit of power.

The paper also compares the efficiency of local and cloud accelerators. We cover that part in The energy case for local AI, with our caveats.

What "local" means in the study

The study counts a model as local if it has up to 20 billion active parameters. The models tested include Gemma 3 at 1B, 4B and 12B, Qwen3 from 4B to 32B, IBM Granite 4.0 and OpenAI's gpt-oss. The 71.3% for 2025 came from gpt-oss-120b, a mixture-of-experts model with 120 billion parameters in total, running on an Apple M4 Max with 128 GB of unified memory.

That is a big machine. Base Drive, which is coming soon, carries two much smaller models: LFM2.5 1.2B, which needs 4 GB of RAM, and Gemma 4 E2B (about 2B parameters), which needs 8 GB. We read the study as evidence about local models as a class, and about how fast that class is improving. Its numbers come from models other than the ones we plan to ship. For more on sizing, see How much memory a local AI model needs.

Where local models do well, and where they struggle

Coverage varied by field. On the two exam-style sets, the best mix of local models covered more than 93% of questions in creative and humanities fields and 60% in technical fields such as architecture and engineering. The authors' summary is that local models "handle most conversational and knowledge-recall tasks while complex specialized reasoning still benefits from frontier capabilities."

The headline numbers cover single-turn queries only. The authors write that multi-turn conversations, agentic workflows, tool use and long-context applications "represent substantial portions of real-world LLM traffic," and those sit outside the main study. An appendix runs a smaller multi-turn check that the authors say shows the same pattern. For long back-and-forth chats, or an assistant that takes actions for you, this study is thinner evidence.

Downloads point the same way

Hugging Face's State of Open Models, Summer 2026 (opens in a new tab), published August 14, 2026, looks at what people pull from the Hub.

• Among models that declare a parameter count, those under 1B account for 83% of all-time downloads, and those over 100B account for 1%.
• Repositories declaring the GGUF library rose 464% over seven months of 2026. GGUF is the llama.cpp file format that local apps such as Ollama (opens in a new tab) load.

These figures show where people are pointing their own hardware. They measure interest, so we pair them with the Stanford results for answer quality.

What our benchmarks page shows

Our benchmarks page (opens in a new tab) uses one source: Artificial Analysis screenshots dated May 30, 2026. On its Intelligence Index chart, scores run from 8.5, the lowest shown, to 45.8 for the best cloud system.

The page marks small local models at about 15, labeled "the class that runs on a box." We placed that marker there as a rough estimate for small models as a group, without measuring it.

Our benchmarks page puts the gap this way: "bigger systems win the hardest tests, but local AI is still useful for private drafts, summaries, search, and focused work where ownership matters." The Stanford results and our chart agree. A small local model handles most everyday single-turn questions, and the largest cloud systems still lead on the hardest ones.

Our fifth belief covers those: routing sends a question to a bigger model when it needs one. Our third belief adds that this should happen only when the owner approves it. On Base Drive, outside help of that kind is a planned addition, designed to stay off until you turn it on. Gemma Gemini routes automatically, without an approval step, but it shows the split message by message, with a route trace on every reply that says whether Gemma 4 E2B on a Jetson or a cloud model answered, and why.

Confidently, fluently wrong

Our terms (opens in a new tab) say every model on our site can be "confidently, fluently wrong." It can invent facts, citations, quotes, numbers, names and law. Small models running on your own hardware are more prone to this than the largest cloud systems, "and that trade is part of what you are choosing."

Here is what that means for the everyday jobs a small model suits:

• Drafts. This is the safest job. You read every line before it goes anywhere, so mistakes get a second look. Check any fact the model added on its own.
• Summaries. Keep the source open. Check names, numbers, dates and quotes against it, since those are exactly the details our terms warn a model can invent.
• Search. Treat a small model's answer from memory as a lead to check. On Base Drive, web search is a planned addition, so for now a factual answer comes from what the model learned in training, with no source attached.

The quickest way to judge all this is to try a small model yourself. On /try (opens in a new tab), LFM2.5 1.2B answers live on a Jetson Orin Nano Super, the board Base Router is built on. Ask it what you would ask a chatbot today, then check the answer. Three things you can try on offlinebase.com without an account covers the other demos.