Local LLM or Cloud API: What Each Really Costs a 100-Person Company in 2026

Short answer (October 2026). For a 100-person company, cloud language models are cheaper than most people expect. In our two usage scenarios, a light office assistant for 70 users costs $16 to $353 a month at list prices, and heavy work with documents and agents for 100 users costs about $100 with a small model and about $2,000 with a mid-tier model such as Claude Sonnet 5.5 or OpenAI's gpt-6.1-sol; EU processing adds about 10%. A dedicated GPU that can run a strong open model costs €1,197 a month rented from Hetzner, €2,093 from Scaleway, or about €505 a month for a bought RTX PRO 6000 written off over three years, plus the server around it and the engineering time to run it. A local model therefore rarely wins on price alone at this size. It wins when data must not leave your control, when volume is high and steady, or when an open model is good enough for the task; for everything else, a cloud API with a data processing agreement and an EU endpoint is the simpler choice.

This article is for IT leaders who need a number, not a principle. It complements our earlier post on local LLMs and cloud APIs for sensitive data with current prices and a worked calculation.

Two usage scenarios

There is no reliable public figure for how many tokens a knowledge worker uses per day, so we calculate with two explicit scenarios. Adjust them to your own logs.

• Scenario A, light assistant use: 70 of 100 employees use a chat assistant 20 times a working day, with about 3,000 tokens of input (question plus context) and 500 tokens of output. That is 88 million input and 15 million output tokens a month.
• Scenario B, documents and agents: all 100 employees work with a RAG assistant and agents 40 times a day, with about 8,000 input tokens (retrieved passages, instructions, history) and 800 output tokens. That is 672 million input and 67 million output tokens a month.

Software developers using coding agents are a separate budget: Anthropic's documentation for Claude Code puts the typical cost at $150 to $250 per developer per month, with an average of about $13 per active day.

What the cloud costs

List prices per million tokens on 12 October 2026, input / output:

Four details change these numbers in practice:

• Caching and batch processing. Context that repeats, such as long instructions or a document set, costs a fraction when cached: for Claude Sonnet 5.5, a cache hit costs 5% of the input price. Requests that can wait are 50% cheaper through the batch APIs of Anthropic, OpenAI, Google and Mistral.
• Tokens are not equal across vendors. Anthropic notes that its tokenizer for Claude 4.7 and later produces about 30% more tokens for the same text. Compare the cost of your own requests, not the price per token.
• EU processing costs extra. OpenAI's EU data residency adds 10% for models released from March 2026; Claude through regional endpoints on AWS Bedrock or Google Vertex AI carries a 10% premium, and Anthropic's own API offers global or US-only processing.
• Prices move. Google's Gemini 3.8 Flash doubles in price on 1 January 2027, and Mistral's newest large model is listed at a promotional price. Budget with a margin and keep the model switchable.

What an own model costs

The model. Strong open-weight models now fit on a single GPU. OpenAI's gpt-oss-120b, released under the Apache 2.0 licence, runs on one 80 GB GPU; the smaller gpt-oss-20b fits into 16 GB. Qwen 3.5 and Mistral Small 4 are also under Apache 2.0, Gemma 4 targets workstations, and DeepSeek V4 uses the MIT licence. One trap for European companies: Meta's licence for its multimodal Llama 4 models does not grant rights to companies with their principal place of business in the EU.

The hardware.

• Rent in the EU: Hetzner's GEX131 with a 96 GB RTX PRO 6000 costs €1,197.30 a month plus €599 setup; Scaleway's H100 with 80 GB costs €2.87 an hour, about €2,093 a month.
• Buy: the RTX PRO 6000 with 96 GB was available in German retail from about €15,490 in September 2026, which is €430 a month over three years. It draws up to 600 W; at the 17.2 cents per kWh that small and medium industrial customers pay in Germany in 2026, according to BDEW, full load around the clock adds about €75 a month. The server, storage, network and a second card for redundancy come on top.

The capacity. Scenario B averages about 1,200 tokens per second during working hours, with peaks several times higher. In SemiAnalysis's InferenceX benchmark, gpt-oss-120b on one H100 reached about 5,900 tokens per second per GPU at 100 tokens per second per user. One well-configured GPU covers a company of this size for text; peaks, long documents and a fallback argue for two.

The part that is not on the invoice. Someone has to install and update the inference server and the models, monitor them, patch the operating system, manage access, keep logs and test each new model before switching. In the cloud price that work is included. Plan it explicitly, as hours of an engineer per month, before comparing totals.

Quality and speed

Price per token is not the whole comparison. The largest cloud models are still ahead on difficult reasoning, long documents and complex agent tasks, and they are fast. In our own RAG project for accountants, the same code ran against a cloud API or entirely on local open models; locally, answers came more slowly and read less fluently, but no document left the client's server. For extraction, classification, routing and RAG answers with sources, a good open model is often enough; for an agent that plans multi-step work, test before you decide.

Data protection

• The German data protection authorities wrote in their 2024 guidance on AI that technically closed systems are preferable from a data protection point of view, and that "open" cloud systems carry risks of secondary use and transfers to third countries. "Closed" means under your control and without training on your inputs, which a cloud service with the right contract and settings can also be.
• A cloud model needs a data processing agreement, settings that exclude training on your data, and a clear answer on where data is processed and stored.
• Transfers to the US currently rest on the EU-US Data Privacy Framework, which the EU General Court upheld in September 2025; an appeal (C-703/25 P) is pending before the Court of Justice. Keep standard contractual clauses in place as a fallback.

When an own model makes sense

A common outcome is a hybrid: sensitive tasks such as contracts, health or tax data on an own model, everything else through a cloud API, and a gateway in between that routes each request, logs it and enforces the rules. The gateway also keeps you free to switch providers when prices change, which in 2026 they do every few months.

Checklist

1. Measure your real token volume for a month before buying anything.
2. Price your own requests with two or three providers, including caching, batch and EU processing.
3. Classify data: what may go to a cloud API under contract, what must stay with you.
4. Test an open model on your own tasks against the cloud model you would otherwise use.
5. If you run your own model, plan operations time, redundancy and model updates.
6. Check licences of open models for your use and location.
7. Put a gateway in front of every model so you can switch.

How we can help

We build custom RAG applications and agentic AI applications that run on cloud APIs, on open models on your own or rented EU hardware, or on both, and we set up the infrastructure and operations for self-hosted models.

FAQ

Is it cheaper to run an LLM locally than to use a cloud API?

For a 100-person company usually not on price alone: heavy use of a mid-tier cloud model costs about $2,000 a month at list prices, while a dedicated GPU costs €1,200 to €2,100 a month rented, plus the engineering time to operate it. A local model pays off with high, steady volume or when data must stay under your control.

What hardware do we need to run a good open model?

A single GPU with 80 to 96 GB of memory runs models such as gpt-oss-120b. For redundancy and peaks, plan two. Smaller models such as gpt-oss-20b run on 16 GB.

How much does a cloud LLM cost per employee?

In our scenarios, from a few cents to about $5 a month per user for light assistant use, and from about $1 to $20 a month per user for heavy work with documents and agents with a small or mid-tier model. Coding agents cost more, around $150 to $250 per developer per month.

Can we use Llama 4 in the EU?

Not the multimodal Llama 4 models: Meta's licence does not grant rights to companies with their principal place of business in the EU. Other open models, such as gpt-oss, Qwen 3.5 or Mistral Small 4, are available under Apache 2.0.

Is a cloud LLM API compatible with the GDPR?

It can be, with a data processing agreement, no training on your data, a clear processing location, and a legal basis for any transfer outside the EU. For highly sensitive data, many companies prefer an own model.

Do EU data residency options cost more?

Usually about 10%: OpenAI's EU processing for newer models and Claude's regional endpoints on AWS Bedrock and Google Vertex AI both carry a 10% premium.

Sources

• Anthropic, Pricing and Data residency, checked 12 October 2026
• OpenAI, API pricing and Your data, checked 12 October 2026
• Google, Gemini API pricing; Mistral AI, pricing, checked 12 October 2026
• Anthropic, Claude Code costs
• OpenAI, gpt-oss-120b model card; Meta, Llama 4 Acceptable Use Policy
• Hetzner, price adjustment for GPU servers, June 2026; Scaleway, GPU pricing
• BDEW, Strompreisanalyse August 2026; NVIDIA, RTX PRO 6000 Blackwell
• SemiAnalysis, InferenceX: gpt-oss-120b on H100
• Datenschutzkonferenz, Orientierungshilfe KI und Datenschutz, May 2024

Categories

About the author

Pavlo Yablonskyi

CTO & Co-Founder

View full profile

CTO & co-founder at Software Development Hub. Software engineer with 20+ years of experience. Python/Django-geek, software architect and IT team leader. Staying up-to-date with tech trends. Strong technical skills and diverse expertise in software structure design, development, team management and cybersecurity.

View full profile