We were told we need a custom model.

You might, and the two cases that genuinely require one are data residency and economics at volume. If the reason given was that the model does not know your domain, that is a retrieval problem and considerably cheaper. It is worth asking whoever advised you which of the four levels they meant.

Is fine-tuning not more accurate than retrieval?

For format, tone and narrow classification, often yes. For factual knowledge, usually no: retrieval can be updated when a fact changes, and a fine-tune has to be redone. Choosing tuning for knowledge is a common and expensive mistake.

What happens when a better model is released?

With an evaluation set, you run it and see. Without one, you are choosing on the strength of a launch announcement. This is the main reason the evaluation set is the first deliverable rather than a later refinement.

Can we keep our data in our own environment?

Yes, and that is one of the two reasons that genuinely justifies self-hosting. It is an infrastructure and operations commitment rather than a modelling one, and it should be entered with that understood.

How much cheaper is level one than level four?

We will not put a figure on it here. What is safe to say is that the levels differ by orders of magnitude in commitment rather than by percentages, and that the cheapest two answer most requirements. The specific comparison for your case is a first-conversation question.

LLM Development

Whether you need a custom model at all, and what to do instead in the nine cases out of ten where you do not.

Most work described as LLM development is prompt, retrieval and evaluation work on somebody else's model. Knowing which level you need separates a project from a research programme.

Talk to a senior engineer Broken Workflow Assessment

Nineteen years of delivery, across six countriesThe evaluation set, in your possession

Four requirementsOne needs a custom model

1Prompt2Retrieve3Tune4Host

Three stop at prompt or retrieval. The fourth runs to the end of the scale.

Illustrative of the ratio the page argues for, not a measured figure.

Four situations

Four situations, and only one of them needs a custom model.

Only the third and fourth justify a custom or self-hosted model. The first is usually a retrieval problem and the second a schema problem, both weeks rather than months.

A general model does not know your domain
It answers plausibly and imprecisely about your products, your rules or your terminology, because none of it was in the training data.
Output format has to be exact
Downstream systems need structure. A model that is usually well-formed fails a strict consumer, and the retries cost more than the calls.
Cost or latency is prohibitive at your volume
The quality is fine and the economics are not, which is a genuinely different problem from a quality problem.
Data cannot leave your environment
A real constraint, and often the only one that actually justifies self-hosting or training.

The four levels

Nearly every requirement lands on level one or two.

This is the decision the whole page exists for. The levels are ordered by cost and irreversibility, and each is genuinely harder to unwind than the one before it.

Nearly every requirement lands here

Prompt and structure
Instructions, few-shot examples, enforced output schema
When it is rightThe model can do the job and is doing it inconsistently. The most common answer
What it commits you toAlmost nothing. Reversible in an afternoon
Retrieval
Grounding answers in your own approved sources at query time
When it is rightThe model lacks your knowledge, not your capability. The second most common answer
What it commits you toA source pipeline and an ownership model. Real but modest, and portable
Fine-tuning
Adapting a model's behaviour on your examples
When it is rightFormat, tone or a narrow classification must be reliable and prompting has genuinely plateaued
What it commits you toA labelled dataset you must maintain, and re-tuning each time the base model moves
Self-hosting or training
Running or building the model in your environment
When it is rightData residency, cost at high volume, or latency that hosted inference cannot meet
What it commits you toInfrastructure, evaluation and an ongoing engineering commitment. The least reversible choice available

ReversibleHard to undo

Two things follow. Fine-tuning is often attempted before retrieval has been tried properly, which is expensive. And level four is an operations decision, not a modelling one.

What we take on

Six things, across the levels.

What the work is at each level, and the one deliverable that sits underneath all four.

Prompt architecture and enforced output
Structured output validated against a schema, with retry and fallback behaviour when a response does not conform.
Retrieval over your approved sources
Grounding, with provenance shown and a defined way to decline when the sources do not cover the question.
Evaluation sets and scoring
Real inputs with expected outcomes, so a model or prompt change becomes a measured comparison. This is the asset that outlives every other decision.
Fine-tuning where it is genuinely warranted
With the dataset, the maintenance cost and the re-tuning trigger stated before starting rather than discovered later.
Self-hosted deployment
Where residency, cost or latency requires it, including the monitoring and update path that makes it operable.
Cost and latency control
Model routing, caching and token budgets set against the job, so spend and response time stay predictable as usage grows.

Level two cannot work over sources nobody owns.

See data engineering

How the work runs

How the work runs, step by step

Five steps in order, with the evaluation set built before anything is changed.

Define the task and the acceptance standard
What the model must produce and how a correct output would be recognised. Without this no level can be evaluated against another.
Build the evaluation set before changing anything
Twenty to fifty real cases with expected outcomes. This is the instrument; everything after it is a measurement.
Exhaust level one, then level two
Properly, not briefly. A meaningful share of requirements stop here, and stopping here is the cheapest good outcome available.
Only then consider tuning or hosting
With the maintenance commitment stated. A fine-tune with no owner becomes stale the first time the base model is updated.
Hand over the evaluation set and the reasoning
So the next decision, including replacing the model entirely, is a measured one your team can make without us.

Evidence

Where this work has been done

Evaluation is the foundation of every level on the ladder above, and it is already how Viithiisys operates AI in production.

Fitelo, evaluation and cost controls in production
Directly relevant: evaluation and cost control are exactly what steps one to three above establish. In place rather than added after an incident, which is the difference between a system a team can change confidently and one nobody wants to touch.
The evaluation set is what turns a model change into a measured decision, and it is the asset that outlives every other choice described on this page.
Conscious Chemist, a customer-facing product experience
A documented 38% faster product-question response.
Evidence of customer-facing AI work, not a model result.
Nineteen years, six countries, since 2007
Nineteen years of delivery, across six countries.

See all case studies

Which page you want

Four pages touch language models. Here is which one you want.

The overlap here is the densest in the page set, so it is worth being explicit rather than leaving the buyer to infer it from titles.

Which of the four levels do we needThis page · you are hereThe model-level decision and the work at each level
Build a product feature using generationGenerative AI DevelopmentThe application around the model, rather than the model choice itself
Should we use generation for this work at allGenerative AI ConsultingThe decision before any build, including the case for retrieval or for neither
It is live and we cannot tell whether it still worksMLOps & LLMOpsOperating and monitoring after release
The sources are unreliable or contradictoryData EngineeringLevel two cannot work over sources nobody owns

What you keep

What you are left holding.

Portable artefacts rather than a dependency on us.

The evaluation set, in your possession
The most durable asset from this work. It makes every future model change a measured decision, including a change away from whatever we recommended.
The acceptance standard, written down
Portable across models and providers, which matters in a category where the leading option changes within months.
A stated level, with its reason
Including what was tried and rejected. A later escalation to fine-tuning should be evidence-driven rather than a fresh guess.
No provider lock-in beyond your own choice
Prompts, schemas and evaluation sets are portable. Where self-hosting was chosen, the update path is documented.

Start here

Tell us what the model is getting wrong, and how you would know it was right.

The second half is the harder question and the one that decides which of the four levels you need. Most requirements stop at the first two.

Talk to a senior engineer Broken Workflow Assessment

FAQ

Questions we are actually asked

Pick a topic, or ask us directly. We answer every inbound within one business day.

Still have questions?

Talk to a senior engineer, not a bot.

Talk to us

Whether you need a custom model at all, and what to do instead in the nine cases out of ten where you do not.

Four situations, and only one of them needs a custom model.

A general model does not know your domain

Output format has to be exact

Cost or latency is prohibitive at your volume

Data cannot leave your environment

Nearly every requirement lands on level one or two.

Prompt and structure

Retrieval

Fine-tuning

Self-hosting or training

Six things, across the levels.

Prompt architecture and enforced output

Retrieval over your approved sources

Evaluation sets and scoring

Fine-tuning where it is genuinely warranted

Self-hosted deployment

Cost and latency control

How the work runs, step by step

Define the task and the acceptance standard

Build the evaluation set before changing anything

Exhaust level one, then level two

Only then consider tuning or hosting

Hand over the evaluation set and the reasoning

Where this work has been done

Fitelo, evaluation and cost controls in production

Conscious Chemist, a customer-facing product experience

Nineteen years, six countries, since 2007

Four pages touch language models. Here is which one you want.

What you are left holding.

The evaluation set, in your possession

The acceptance standard, written down

A stated level, with its reason

No provider lock-in beyond your own choice

Tell us what the model is getting wrong, and how you would know it was right.

Questions we are actually asked