Frequently Asked Questions

A few months back a client in the legal sector asked us for something very specific: they wanted an assistant to help their team search through thousands of pages of case files, but they flat-out refused to let any of those documents leave their server. Not to OpenAI, not to Anthropic, not to anyone. That’s exactly where the difference between “cloud” AI and self-hosted open-source AI comes in.

When you talk to ChatGPT or Claude, your questions travel to a server run by OpenAI or Anthropic, get processed there, and the answer comes back to you. Convenient, fast, and you never have to think about infrastructure. The catch is that for some businesses, sending sensitive information to a third party simply isn’t an option — whether because of a contract, industry regulation, or the client just not trusting it.

That’s where open-source (or “open-weight”, the more precise term) models come in: Meta’s Llama, Mistral, Qwen. Models you can download and run on your own hardware, so not a single word of what you ask ever leaves your network. With tools like Ollama or vLLM, setting one up isn’t nearly as complicated as it sounds — a couple of days of work, not weeks.

What does cost real money is the hardware. For the law firm’s project we ended up renting a GPU machine from a cloud provider (buying your own graphics cards is something we almost always advise against), and even then the monthly bill ran about four times what using the Anthropic API would have cost for the same volume of queries. And honestly, the quality of the answers was worse: Llama does a decent job, but it’s not in the same league as Claude or GPT-4 on complex reasoning over long text.

So the real question isn’t “open source, yes or no”. It’s “which matters more to me, confidentiality or quality”. For most of our clients the answer is clear: using a serious provider’s API, with a solid data processing agreement in place, ends up cheaper, faster to set up, and gives better results. The law firm was one of the rare cases with no way around it: their own lawyers had confidentiality obligations that ruled out any middleman, however good the contract on paper.

There’s a middle ground people don’t talk about as much: some providers (Azure OpenAI, Amazon Bedrock) offer the same closed models but with contractual guarantees that your data isn’t used for training and isn’t shared with anyone else. For a good number of clients worried about privacy, that solves the problem without standing up your own servers or giving up quality. We’ve yet to explain this to someone without watching their shoulders drop a little.

What I honestly don’t know is what happens in two years, once open models close the quality gap with the closed ones. Maybe “self-hosted or not” stops being a question of sacrificing quality and turns into a purely operational cost calculation. Or maybe the big models just stay one step ahead forever. We’ll see.

→ Direct link to this question

A client who resells secondhand industrial machinery asked us a couple of months ago if we could “train an AI” on fifteen years of emails and quotes sitting on a server. He wanted an assistant that could answer his sales reps’ technical questions using that history. I told him something like that was doable, but “train” was the wrong word, and that if anyone offers to “train your own model” for two thousand euros, he should be suspicious.

Training a model from scratch, or even fine-tuning an existing one, takes a lot more data than a typical small business has lying around, and the result usually isn’t any better than the cheaper option: giving the model access to your documents at the moment someone asks a question, instead of trying to get it to memorize them. That’s called RAG (retrieval-augmented generation), and it’s what almost any agency actually builds when they talk about “AI with your company’s data,” even if they don’t always call it that.

The practical difference is this: with fine-tuning you’re modifying the model’s weights, which means GPU time, training runs, and having to retrain every time your data changes. With RAG you store your documents, chunked up, in a vector database, and when someone asks something, the system pulls the most relevant chunks and hands them to the model as context before it answers. Adding a new quote is literally just uploading a file. No retraining, no GPU bill.

For the machinery client we built exactly that: their emails and quotes chunked and dropped into a vector database (we used pgvector since they already ran Postgres, no need to bolt on a new tool), plus an internal assistant a rep can ask “what did we quote client X for that forklift model back in 2024?” and get the relevant document back with the answer already summarized. It cost a fraction of what fine-tuning would have, and we had it in production in two weeks.

There is one case where fine-tuning actually makes sense: when you need the model to always answer in a very specific format or tone and RAG can’t force that on its own (say, generating code in a particular internal style based on thousands of existing examples). But that’s rare in the kind of work we do for small agency clients, and when it has come up it’s almost always been for a very specific content-generation case, not “answer questions about my data.”

What bugs me most about this whole topic is how many proposals I’ve seen floating around offering to “train your own custom AI” when what’s actually being built is a RAG setup with a handful of lines of code wrapped around it. It’s not exactly a lie, but it inflates the price and makes people think they need far more infrastructure than they actually do. If someone pitches it that way, just ask directly: is this fine-tuning or RAG? The answer tells you pretty quickly whether the person selling it to you knows what they’re doing.

→ Direct link to this question

A client emailed us a couple of weeks ago asking for “a chatbot to handle orders.” Ten minutes into the call it was clear that wasn’t what they actually wanted. They wanted something that checked stock, worked out shipping, and, if needed, pinged the supplier. That’s not a chatbot anymore. That’s an agent.

The short version: a chatbot replies. It gets a message, whichever model is running generates an answer, and the loop ends there. An agent can decide to do something before it answers: query a database, call an API, run a script, and only then give you a response that actually accounts for what it found. The key isn’t the model itself, it’s that the model has access to tools and can choose to use them.

This is called “tool calling” or “function calling,” and it’s been around for a while. What’s changed in the last few months is that it’s become reasonably reliable. It used to make up parameters or call the same API three times for no reason. With Claude Opus or Sonnet, or GPT-4.1 onward, it fails a lot less. Not never. It still happens, especially with poorly documented tools or ambiguous parameter names.

For this client we built something fairly simple: an agent with three tools. Check stock, calculate shipping cost by weight and postcode, and draft a supplier alert when stock dropped below a threshold. No free-form language in the critical part: the agent decides which tool to call, but the shipping calculation is plain code, not something the model “reasons” its way through. Watch out for this one, because a lot of people skip it on their first agent build: the model decides the what, the code handles the how. Let the model guess shipping rates and you’ll get surprises —annoyingly hard to debug ones too, since the model won’t explain why it got it wrong, it just returns a different number each time.

The hard part wasn’t the AI. It was deciding what happened when the agent picked the wrong tool or got stuck halfway, say on a timeout from the supplier’s API. We had to add retries and, more than anything, a step limit so the agent wouldn’t loop calling the same tool twenty times if something broke. The first version we tested didn’t have that limit. In an internal test it sat there querying stock in a loop for almost two minutes before we killed it by hand.

If you’re weighing something like this for your business, the question that actually matters isn’t whether AI can do it (it almost always can, at least in a demo). It’s what happens the day the tool it calls fails, returns something odd, or takes fifteen seconds to respond. That part isn’t solved by the model. You solve it.

→ Direct link to this question

The first time a client asked me “wait, why are you charging me per word?” I had to stop the meeting and explain that it’s not words. It’s tokens, and the difference matters more than it sounds like once the API bill lands at the end of the month.

A token is a chunk of text, almost never a whole word. “Cat” is usually one token. “Extraordinarily” might split into three or four. Models from OpenAI, Claude or Gemini don’t read letters or loose words: they read these fragments, and every API call gets billed on how many go in (the prompt you send) and how many come out (what the model replies). Spanish, as it happens, tends to burn more tokens than English to say the same thing — we found that out the hard way on a hospitality client’s project, when the monthly API spend came in almost 40% over what we’d estimated using English test copy.

That has consequences that shape how you design an AI feature, not just what it costs:

• A long system prompt (the fixed instructions you send the model on every request) gets billed on every single call, even if the user only typed two words.
• If a chatbot keeps the full conversation history to fake “memory,” every new message drags the whole thread along with it, so the cost per turn climbs as the chat goes on.
• The more capable models don’t just charge more per token — left unconstrained, they also tend to write longer replies, so the cost stacks twice over.

When we budget an AI integration for a client, the first question isn’t “how much will AI cost” in the abstract anymore. It’s estimating tokens per interaction. We take real examples of what the user is going to type, run them through each provider’s tokenizer (they all publish one, free to use) and multiply by the expected monthly volume. It’s a tedious step. It’s also the one that stops surprises.

There’s a detail almost nobody asks about that more people should: a model’s context limit is also measured in tokens, not “messages” or “pages.” If a client wants the AI to “read the whole product catalog” before answering, that catalog has to fit, in tokens, inside that window. If it doesn’t, you either chunk it up or build a vector store that pulls in only what’s relevant per query — and at that point it’s a different project with a different budget, not a box you tick in the first meeting.

What still bugs me is that almost no provider shows you a live token count while you’re typing the prompt, as if they’d rather you not think about it until the charge hits the card.

→ Direct link to this question

MCP stands for Model Context Protocol, an open standard that Anthropic started popularizing and that several AI assistants now support. The idea, stripped of marketing: instead of coding a different integration every time you want an AI to talk to Trello, a database or an image generator, you define an “MCP server” once, and any compatible assistant knows how to use it.

Before this existed, connecting an assistant to an external tool meant writing custom code: authentication, request formatting, error handling, all different for each service. With MCP, whoever maintains the server, whether that’s the service’s own company or someone in the community, handles that part once, and the assistant just needs to speak the protocol.

In practice, an MCP exposes a list of concrete actions to the assistant: read a Slack channel, create a Trello card, generate an image in a given style. The assistant decides when to use each one based on what you ask it, the same way you’d decide which button to click in that tool’s interface if you were using it by hand.

Not every MCP is equally mature. Some have been polished for months and run without surprises, and some are freshly published and fail over silly things like a badly explained quota limit or a response in a format you weren’t expecting. Before giving write access to an MCP on something sensitive, a real project board, an email account, it’s worth testing it first on something that doesn’t matter if it breaks.

If you’re working with an agency or a technical team that says it uses MCP, the question that actually matters isn’t what it is, it’s what it has access to and with what permissions: reading isn’t the same as writing, and an assistant with write access to a real tool can mess up just as much as an unsupervised new hire would.

→ Direct link to this question

A couple of months ago a developer on the team spent three hours looking for a JavaScript library method that, according to ChatGPT, existed. It didn’t. It sounded so plausible — something like Array.prototype.groupByAsync — that nobody checked it until the build broke. That’s a hallucination: the AI doesn’t say “I don’t know,” it makes up an answer with the same confidence as if it were true.

And it’s not a rare bug that only happens to a mediocre model. It happens to all of them, including the good ones. A language model doesn’t “know” things the way a database does: it predicts the most likely next word based on what it saw during training, and when the topic is very specific or poorly documented, that prediction can sound perfect on the outside and be empty on the inside.

What we’ve learned at the agency, more than distrusting AI as a whole, is distrusting it selectively. When I ask a model for a verifiable fact — a library version, the exact name of a method, a figure — I treat it as a hypothesis, not a fact. If I can check it in the official documentation in 30 seconds, I check it. If I can’t, it doesn’t go into the client’s code. Full stop.

There are patterns that help catch it before it’s too late. The more niche or recent the topic, the more likely the invention: a library released two months ago is fertile ground for the model to fill gaps with something plausible. If you ask the model for the exact source of what it just claimed and the answer is vague, or changes between attempts, that’s a bad sign. And when it generates code, you run it, always. A broken import gets caught in two seconds by running the project; eyeballed in a quick review, it slips through without anyone noticing.

With non-technical clients this is trickier, because many assume that if the answer “comes from AI” and sounds coherent, it’s correct. I explain to them that a customer service chatbot that makes up a price or a returns policy isn’t a cosmetic glitch, it’s a real legal and trust problem. That’s why when we build something like that we almost always tie it to the company’s real data (with document retrieval or heavily restricted prompts) instead of letting the model “remember” the information from training.

What I still haven’t fully figured out is how to explain this risk without the client ending up distrusting the whole tool. Saying “it can make things up” sounds like it doesn’t work, and that’s not it: it works very well for almost everything, except for the data it can’t afford to get wrong. That nuance is the hardest part to get across in a sales meeting, and I still haven’t found the sentence that sums it up without sounding like a legal disclaimer.

→ Direct link to this question

When I started building the first AI chatbot for a client, I assumed the model “remembered” the conversation the same way a person would. It doesn’t, and understanding that changes quite a bit about how you design this.

A language model doesn’t store anything between one question and the next by default. Every time you send it a message, you’re actually sending it the entire previous conversation again (or a summary of it) along with the new message. What we call “memory” is, in most of the chatbots we use daily, simply resending the full history every time you write something.

That has a practical limit: the longer the conversation, the more text has to be resent, and models have a cap on how much text they can process at once, the so-called “context window.” For long conversations, some tools summarize the old parts and keep only the recent bits verbatim, so they don’t run out of space.

Then there’s real memory, the kind that persists across separate conversations, an assistant remembering something you told it last week. That isn’t magic from the model — it’s a separate database where specific facts are stored (your name, a preference, a detail about your company) and then injected into the model at the start of every new conversation, like a cheat sheet.

For a customer service chatbot, this translates into a very concrete design decision: which customer data is worth storing and re-injecting (their name, an order’s status) and which doesn’t need to be, and shouldn’t be, kept permanently. Storing too much is also a privacy risk, not just unnecessary technical overhead.

The first time a client asked us why the chatbot didn’t remember what it had been told the day before, I had to explain that the answer wasn’t a bug — it was that nobody had asked the system to keep that information from one day to the next. It can be done, but it has to be designed that way on purpose.

→ Direct link to this question

These three terms get used as synonyms in sales meetings, and they aren’t, so it’s worth clarifying before someone sells you “AI” when it’s actually a script with three if-statements.

Automation, in the classic sense, is getting a computer to repeat a task by following fixed rules a person wrote: if an email arrives with the word “invoice,” move it to this folder. There’s no learning or adaptation, just explicit rules. It’s the cheapest and most underrated of the three, and it solves most of a small business’s repetitive problems without needing anything more sophisticated.

Machine learning is a step up: instead of writing the rules by hand, you give the system lots of examples and it “learns” the pattern. A modern spam filter doesn’t have a list of banned words written by a person — it has seen millions of emails marked as spam or not spam and worked out the pattern on its own.

Artificial intelligence is the umbrella that covers all of the above, including machine learning, but also more recent things like language models (LLMs) that generate text, images or code. Not all AI is machine learning in the strict sense, and certainly not all automation needs AI of any kind.

What I see in practice, working with small businesses, is that most of what people call “I want to add AI to my company” is actually solved with simple, cheap automation: connecting a form to a CRM, sending automatic reminders, sorting emails by type. Generative AI comes in afterward, for whatever genuinely requires generating content or understanding natural language with nuance.

Before asking for “AI” for your business, ask yourself whether what you actually need is a well-placed fixed rule. It’s cheaper, it fails less, and most of the time it’s exactly what was needed.

→ Direct link to this question

A couple of months back a client who sells clothes online asked if we could generate all forty product shots for the next season with AI instead of booking the photographer again. It would have saved him something like €3,000 and two weeks of waiting. I told him yes, technically it’s doable, but “legal” and “no problems” aren’t the same question, and answering it properly took more digging than I expected.

The copyright part turns out to be simpler than people think: in Spain and the EU, something generated entirely by AI without human creative input doesn’t get copyright protection, nobody counts as the “author” in the legal sense. That doesn’t mean it’s automatically public domain or free to use however you like, it comes down to the tool’s terms of service. Midjourney, for instance, hands commercial ownership to the paying user; other tools keep it, or share it. Before we drop a single generated image onto a client’s site, I check that specific tool’s terms. I don’t assume they all work the same way.

But what actually worries me isn’t the intellectual property side. It’s resemblance. I explained to the client that an image generator can spit out something that looks too close to an existing stock photo, to a garment with a registered pattern, or, worse, to a real recognizable person, and that’s where you risk infringement or image rights, no matter what the tool’s fine print claims about ownership. Text has the same problem from a different angle: ask a model to write up a product’s technical specs and it invents a certification or a feature the product doesn’t actually have, and now the legal issue isn’t intellectual property at all, it’s misleading advertising, and that lands squarely on the client.

We ended up settling on a middle ground: AI generates the background and composition of the product shots, but the garment itself stays a real photo, composited in, and any generated text goes through a human review pass before it’s published. No inventing features, full stop, because saving on a photographer isn’t worth walking into a lawsuit. We use AI for this blog’s own header images, the code running this site generates them automatically whenever I publish a new post, and I don’t lose sleep over that one because they’re abstract illustrations, not product shots or people, nobody mistakes them for something real.

There’s another angle almost nobody brings up: if a client asks you to never mention anywhere that the content is AI-generated, and someone finds out anyway, the reputational damage usually outweighs whatever the underlying legal risk was. I’m not a lawyer and this isn’t a closed legal verdict, it depends on the tool, the country, and the specific use, but it works as a rule of thumb: the closer the generated thing gets to something or someone identifiable, the closer you are to a problem, legal or just a credibility one.

→ Direct link to this question

Three weeks ago a client who sells custom-made furniture asked me for a quote on an assistant that would answer questions about timelines and materials before the visitor reached the contact form. I gave him the number of development hours and he looked shocked, but not for the reason I expected. It was because I’d just explained that was only the first invoice.

Here’s the mess almost nobody explains upfront: building an AI assistant has a fixed cost (building it, connecting it to your site, training it on your catalog) and a variable cost that depends on how much it’s used, month after month, indefinitely. No expiry date. You pay the fixed cost once and that’s it. The variable one is what catches people off guard.

That variable cost is what the model provider charges — doesn’t matter if it’s OpenAI, Anthropic or Google — per conversation: it’s measured in tokens, roughly chunks of a word, and the more questions your assistant answers and the longer the replies, the more tokens it burns through. For a bot handling twenty queries a day, the spend is trivial, a couple of euros a month. For one that suddenly gets traffic from a campaign and handles two thousand conversations a day, the bill can multiply by forty without anyone touching a single line of code.

It happened to us with a home renovation client. We launched their assistant on a low-traffic site, budgeted a monthly figure with some margin, and three weeks in a social media campaign brought ten times the usual visits. The assistant worked perfectly. The AI provider’s bill spiked. And the client called us worried, thinking something had broken, when in fact everything was working better than expected, which is honestly a good problem to have. Nobody had warned him that scenario existed, though.

Since then we do two things on every project like this. We set spending caps directly in the provider’s dashboard, so it can never spike without anyone noticing. And we pick the model based on what the assistant actually needs to do, not the most powerful one by default: answering “how long does delivery take?” doesn’t require the priciest model on the market. A cheaper, faster one is more than enough, and that shows up clearly at the end of the month.

What I still haven’t fully figured out is how to explain this in the sales meeting without it sounding like “this could get very expensive and we don’t know how much.” Because that’s not really it — with the caps set properly the risk is manageable, it’s just a matter of configuring it beforehand instead of after the scare. But it is true that any fixed quote someone gives you for an AI project with variable usage is, at minimum, incomplete if it doesn’t also talk about that second invoice.

→ Direct link to this question