When Not to Use an LLM

We build AI systems for a living, which is exactly why we spend a surprising amount of time talking clients out of using one. A language model is a specific tool with a specific failure profile — fluent, confident, occasionally wrong — and there is a growing class of software where that profile is disqualifying, no matter how good the model gets.

Here is the checklist we actually use. If your problem hits any of these, reach for something more boring first.

1. The answer is deterministic

Tax on an invoice. Whether a date falls in Q3. Which price tier a customer is on. These have exactly one correct answer, computable by arithmetic or a lookup — and asking a probabilistic text generator to do arithmetic is paying more for a worse guarantee. The failure is not hypothetical: an LLM will compute the total correctly a thousand times and then, without warning or any change in input, transpose two digits on attempt one thousand and one. Code either works or fails loudly. Prefer the thing that fails loudly.

Use instead: a function, a formula, a database column.

2. A query already solves it

“How many orders did we ship last month?” is a SQL query. “Find every contract that mentions termination” is a text search. A remarkable share of proposed LLM features are queries wearing a costume — and the costume adds latency, cost, and a new way to be wrong. The test: if you can write down the exact rule that produces the answer, you do not need a model to approximate the rule.

Use instead: SQL, full-text search, a filter, a regex.

3. Nobody will check the output

LLMs are strongest where a human reviews the result before it matters — a drafted email, a summarized document, a suggested reply. They are weakest where output flows straight into consequences: a price quoted to a customer, a record updated, money moved. If your architecture diagram has model output crossing into the world with no checkpoint — human review, schema validation, a business-rule gate — you have not built an AI feature; you have built an incident, and scheduled it for later.

Use instead: the same model, with a human or a validator between it and anything irreversible.

4. Being wrong is expensive and hard to detect

There are two costs to every error: the damage, and the time it takes to notice. LLMs are most dangerous where both are high — medical, legal, financial, safety, compliance. A wrong answer in a brainstorm costs nothing; a plausible-but-wrong clause in a contract can cost the company quietly, months later. The subtlety is what makes it dangerous: LLM errors do not look like errors. They look like every other answer.

Use instead: retrieval that quotes its sources so a professional can verify in seconds — or keep the model out of the loop entirely.

5. The data cannot leave

If the workload involves data governed by privacy law, client confidentiality, or contractual residency terms, “just call the API” is not an architecture decision — it is a compliance decision someone else will get to make about you, later. This is solvable (self-hosted models, providers with zero-retention agreements, Canadian-region deployment — considerations we covered in our PIPEDA guide), but it must be solved before the pilot, not after the data has already made the trip.

Use instead: the same feature, on infrastructure chosen to match the data’s obligations.

6. The economics only work if the model is free

A feature that runs a model on every keystroke, every row, or every page view has a cost curve that scales with your success. Plenty of viable-in-demo features die here in production. Run the arithmetic early — calls per day times tokens per call times price — and compare it honestly to the boring alternative. We published a cost-and-quality matrix for exactly this exercise.

Use instead: caching, smaller models, batching — or the boring alternative, which is frequently a cron job.

Where LLMs genuinely earn their keep

The point is placement, not scepticism — the same failure profile that disqualifies a model from computing invoices makes it excellent at drafting things humans review, extracting structure from messy documents, answering questions grounded in retrieved sources, and classifying at volumes no team could read. Every good AI system we ship puts the model where fluency is the job and verification is cheap — and keeps deterministic logic in ordinary code around it.

That sorting — which parts of your operation belong in each bucket — is precisely what our AI readiness sprint produces, including the honest finding when the answer is “none of this needs a model yet.” For the other side of the coin, see what AI can actually do for a ten-person business.

Write us a brief if you are weighing one of these calls. We answer within one business day, and “use a spreadsheet” is an answer we are genuinely willing to give.