Every agent I build runs in one of two places. Managed means it runs in a European data centre of one of the big clouds — Azure or AWS — and produces answers through a model provider's API: Anthropic, OpenAI or whichever one wins on your own questions, under a data processing agreement. Self-hosted means the whole thing sits inside your perimeter on a local model and no part of an answer travels over the internet.
The second option is the one people ask about the moment employee records or contracts are involved. Here is what it actually involves.
What "self-hosted" includes
- The language model runs on your machine. No request leaves the building to produce an answer.
- The search index built from your documents lives on the same machine, next to the model.
- Employee records, contracts and policies stay in your storage. The agent reads them; it never copies them anywhere.
- Conversation logs stay with you too, which matters when HR or legal want to review what was said.
- Updates are deliberate: a model version changes when you decide to change it, not when a vendor ships something new.
The hardware
One Mac mini M5 Pro with 64 GB of memory handles a company-wide HR assistant: the model, the index over your policies and the conversation history. That is about €3,100 once, and a few euros a month in electricity. No server room, no rack, no cooling project — it sits on a shelf and answers over your network.
Bigger jobs need more: parsing long contracts or a document flow of thousands of pages a month runs better on two machines, one serving answers and one doing the heavy reading. We size that from your actual volume during the pilot, not from a guess.
What you give up
A model that fits on your own hardware is smaller than the one behind the Claude API, and it shows on hard questions. In my own benchmarks the gap is narrow on structured retrieval — "what does the policy say about carrying vacation over" — and wide on tasks that need careful reasoning over long documents, like reading a 40-page contract and explaining why a liability clause is unusual.
So the honest split looks like this. Self-hosted is enough for employee questions over policies, document recognition, style rules and most internal lookups. The cloud model still wins on contract review and on anything that has to produce polished text for an outside reader.
What GDPR asks, and how each option answers
The managed option gives you two processors: the cloud that hosts the agent — Azure or AWS, European region, with their own GDPR documentation, ISO certificates and audit reports — and the model provider, Anthropic or OpenAI, under a data processing agreement. Your data is not used to train models. In practice that is two lines in your processor register, backed by paperwork both vendors publish themselves, and nobody on your side has to run a server room or argue with an auditor about a cloud they have never seen.
With the self-hosted option there is no third-party processor in the answer path at all. For companies whose works council, insurer or client contracts forbid sending personal data to an external service, that is usually the difference between a project that can start and one that cannot.
How we decide without guessing
Both options run the same agent code. The model behind it is a setting, so the decision is reversible: starting managed and moving in-house later is a configuration change and a hardware purchase, not a rebuild.
In a pilot we run your own questions through both and compare the answers side by side. You see where the local model is indistinguishable and where it is not, with your documents and your wording, before anyone signs off on hardware.
Book a call and we will work out which half of your process can stay in-house.