A Developer Said "Open Source." You Nodded.
A client of mine, an Edmonton property manager, called me last spring after getting a proposal from a developer. He had suggested using an "open source AI model" to keep costs down and give her full control over her data. She wasn't sure if she should be impressed or worried. She didn't actually know what it meant.
The phrase comes up often in vendor proposals and tech conversations now. This post explains what open source actually means in the AI context, which models are open source, and whether any of it changes what you should use.
In AI, It's About the Weights, Not Just the Code
In traditional software, "open source" means the code is publicly available. Anyone can read it, modify it, and build on it. That's what powers WordPress, Firefox, and Linux.
AI adds a specific twist. For an AI model, the more important release is the model weights, not the training code.
Model weights are the billions of numerical values the model learned during training. Think of them as the model's knowledge, encoded in numbers. When a company releases the weights publicly, anyone can download and run that model on their own computers.
Without the weights, releasing the code doesn't give you much. With the weights, you can run the model yourself, fine-tune it on your own data, or embed it into a product you're building.
Which AI Models Are Open Source
A few models are genuinely open source, weights included:
- Llama (Meta): The most widely used open source model family. Llama 3 is capable enough for many business tasks, and there are versions tuned for different levels of computing power.
- Mistral: A French AI company releasing high-quality models, including some designed to run efficiently on smaller hardware.
- Gemma (Google): Google's lighter-weight open model family, built for research and custom deployment.
Well-known closed models are not open source: ChatGPT (OpenAI), Claude (Anthropic), and Gemini (Google's main product). You can use these via a subscription or API, but you cannot download the weights and run them yourself.
What Self-Hosting Actually Takes
Running an open source model yourself requires hardware. Not a laptop. A server with serious computing power, typically one or more high-end GPUs.
Most small businesses don't have that. Renting the computing power from a cloud provider is an option, but it requires technical setup and ongoing management. This is a job for a developer, not a business owner.
The phrase "open source AI" sometimes gets used loosely to mean "we're using a free model." In practice, free to download does not mean free to run.
The Real Reason Privacy-Conscious Teams Care
The main reason businesses consider open source AI is data control. When you send text to ChatGPT or Claude, it travels over the internet to servers you don't own. For most tasks, that's fine.
But if you're dealing with sensitive documents, such as client legal agreements, patient records, or financial data, you may not want that content leaving your building. Running an open source model on your own server means your data stays local.
Some work in Alberta comes with data residency requirements. Healthcare, legal, and government contracts often specify where data can and cannot travel. For organizations in those situations, a locally hosted open source model can be the difference between compliant and not.
Cost Is Not as Simple as "Free"
Open source models are free to download. That sounds appealing. The real cost is infrastructure and the developer time to set it up and keep it running.
A developer charging $5,000 to configure a local AI deployment, plus $300 per month in server costs, is often more expensive than $20 per month for a hosted subscription. The math only flips once you have high volume, strict privacy requirements, or a need to fine-tune a model on your own proprietary data.
When Open Source Is Worth Thinking About
Open source AI becomes relevant if:
- Your data cannot leave your building due to legal, healthcare, or government contract requirements
- You are building a custom product and need to avoid per-token API costs at scale
- You have a developer who can manage the setup and maintenance
- You need to fine-tune a model on sensitive internal data that can't be sent to a third-party API
For most business owners using AI as a day-to-day tool, none of these apply. The hosted subscription is the right choice.
The Honest Part
As of early 2026, the best closed models (Claude, GPT-4o, Gemini) still lead most open source alternatives on general-purpose tasks. Llama 3 and Mistral have gotten remarkably capable, but for writing, summarizing, and answering questions, a hosted closed model is usually the stronger choice.
That gap is narrowing fast. For specialized or high-volume tasks, some open source models already outperform closed ones in specific areas. Rankings change, sometimes quickly.
Open source also carries its own risks. There is no company backing the model with safety updates, usage monitoring, or customer support. The responsibility for those things falls on whoever is hosting it.
The Short Version
Open source AI means the model weights are publicly available, so anyone can download and run the model. The most widely used examples are Llama and Mistral. Self-hosting requires real infrastructure and a developer to manage it.
What this means in practice:
- If you're using AI day-to-day, you're using a hosted product. Open vs. closed is not a decision you need to make.
- If a developer proposes an open source setup, ask what the full cost is (setup, server, maintenance) and compare it to the hosted alternative.
- If your data has strict residency requirements, self-hosted open source is worth evaluating seriously.
- Performance of open source models has improved significantly, but the best hosted models still lead on most general tasks.
If you're trying to figure out whether an open source AI setup is the right call for a project, we can help you work through the tradeoffs. Book a free discovery call and we'll give you a straight answer based on your actual situation.