Reviewed 11 October 2026
Local AI means inference runs on your machine; offline AI means the required workflow continues without network access. A desktop interface can still call a cloud model, a remote search service or a connected tool. Verify the complete route before treating an application as a private data boundary.
Direct answer: LM Studio suits desktop model and document exploration; Ollama suits a programmable local service. Choose the model, context and integration after testing hardware fit and the actual business task. Neither interface guarantees good answers.
Verify what stays local
LM Studio’s offline documentation says downloaded models, local document processing and its local server can operate without internet access. Model discovery, downloads, runtimes and updates need connectivity. Prepare those dependencies before an offline deployment rather than discovering that a required component is missing during a sensitive task.
Ollama’s FAQ distinguishes local inference from cloud-hosted models. It documents a local-only option using OLLAMA_NO_CLOUD=1 and a localhost default listener. Those are useful configuration facts, but a connected front end or external tool can create another network route. Inspect the application stack, not just the model process.
Define the data boundary: which files are loaded, where prompts and outputs are stored, what logs retain and whether backups synchronize elsewhere. A local document index may be included in a corporate cloud backup. A browser extension may send queries to another provider. “Nothing leaves” should refer to a tested configuration and workflow, not a slogan.
For sensitive use, disconnect networking in a safe test and run the complete task. Include document retrieval and any tool calls. Observe attempted outbound connections and failure behaviour. Record the app version, model file, embedding model and configuration. Offline success is different from a compliance determination; apply the relevant organizational requirements to the actual setup.
Fit the model to the hardware
LM Studio’s system requirements recommend at least 16 GB RAM and 4 GB dedicated VRAM on Windows, with AVX2 support for x64. Those recommendations do not promise that every model or context fits. Weights, context cache, runtime overhead and other applications all consume memory. Quantization can reduce memory while changing task quality.
| Pattern | Useful for | Tradeoff to test |
|---|---|---|
| Desktop local app | Chat, document exploration, manual review | User setup, model choice and device resources |
| Local API service | Repeatable integrations and developer tools | Authentication, process ownership and monitoring |
| Dedicated local server | Shared constrained workloads | Capacity, access control, backups and availability |
| Cloud fallback | Tasks that exceed local capability | Changed data boundary and provider dependence |
Start with a smaller model and modest context. Increase them only when a measured task requires it. A model loading successfully does not prove useful latency at peak memory. In Ollama, use its documented inspection commands to check CPU/GPU placement and observe real resource use.
Read the model licence separately from the app licence. Open weights do not automatically mean unrestricted commercial use. Preserve the exact model file and hash or version identifier so an update can be evaluated rather than silently changing accepted behaviour.
Build a document workflow with evidence
Begin with a small, well-defined document collection. Remove obsolete versions or label their validity periods. Keep source access permissions attached to the retrieval process. A local index that mixes departments can reveal information to users who were never authorized to read the original files.
Create questions with known answers and questions that should be refused because the evidence is absent. Require source references and inspect whether they support the answer. Test tables, scanned pages, conflicting policies and text in the languages your team actually uses. A large context window does not ensure correct extraction from all formats.
Separate retrieval failure from generation failure. If the correct passage never reaches the model, changing the chat model may not fix the result. If the passage is present but the answer changes a number or date, inspect the prompt and validation. Keep document identifiers and retrieved chunks available for investigation without exposing unnecessary content in logs.
Do not give a document chat unrestricted write access to operational systems. Start with reading and drafting. For a generated classification or extracted field, validate it against the business schema and route uncertain cases to review. See production RAG design for retrieval and access-control boundaries.
A practical local AI acceptance test
Choose twenty to fifty representative tasks as an initial diagnostic set, not a statistically definitive benchmark. Include ordinary questions, missing evidence, long inputs and difficult formats. Run the same tasks on each candidate configuration and record accepted quality, elapsed time, memory and reviewer effort. Keep all outputs so a future upgrade can be compared.
Test the disconnected workflow, application restart and model reload. For a shared server, test unauthorized access and simultaneous requests. Keep the listener local unless a deliberate network deployment includes authentication and access restrictions. Do not expose a development inference endpoint merely because another laptop needs access.
Budget operating work. A hypothetical workstation costing $1,800 spread over 36 months contributes $50 monthly to a management view. Add power, maintenance, backup and review. Compare that total with the actual alternative at the same quality boundary. Existing hardware can reduce incremental cash cost, but its capacity and opportunity cost still matter.
The next useful step is a fully disconnected trial on one harmless document set. Record exactly which operations stop working without the network. Then decide whether the local route meets the task’s privacy, latency and quality requirements. Use current model selection for broader tradeoffs, but do not substitute a cloud leaderboard for testing a quantized local model.
Plan maintenance for a disconnected environment
An offline deployment still needs an update process. Prepare approved model files, runtimes and application installers in a connected staging environment. Record their provenance and integrity before moving them to the target device under the organization’s process.
Retain the previous accepted configuration so a failed update can be reversed. Re-run the saved task set after changing the model, runtime, quantization or context setting. A new file with a similar name can behave differently or consume more memory.
Consider physical access and shared accounts. Local inference protects against some network transfers but does not prevent another authorized device user from reading chat history or document indexes. Apply operating-system access controls and retention rules to those artifacts.
Finally, test backups and restore. If an index can be rebuilt from protected source documents, record the procedure. If prompts or accepted outputs are needed for work, include them in a deliberate retention plan. Offline operation is a supported workflow that needs ownership, not a device that can be forgotten after installation.
Frequently asked questions
Is a desktop AI app automatically offline?
No. It may use cloud models, search or connected tools. Test the complete configured workflow with networking disconnected.
Does 16 GB RAM guarantee a useful local model?
No. Model weights, context, runtime overhead and other applications determine fit. Measure the intended configuration.
Can local document retrieval leak information?
Yes, if source permissions, index access, logs or backups are misconfigured. Local processing does not remove authorization requirements.
What should be recorded for an upgrade?
App/runtime version, exact model file, configuration and retained acceptance tasks with outputs and resource measurements.
Sources checked 11 October 2026
Research Path