Two very different readers land here: teams who want AI inside a product they already run, and AI companies who need a website that survives technical scrutiny. Both are making the same call, which is whether the people they hire understand what happens after the demo works. Here is how we build language model features that hold up with real users, what actually drives their running cost, and where we would tell you not to bother.
The model is the least durable part of the system
Most AI features outlive the model they launched on. Providers deprecate versions, shift default behaviour between releases, adjust safety thresholds and retire endpoints on their own timetable. If a model identifier is hard-coded in twelve places, each one is a future incident. The parts that last are unglamorous: the prompt stored as a versioned artifact, the schema you expect back, the validation that rejects malformed output, the retry policy, the audit trail. We put the provider behind one interface, with model id, sampling parameters and system prompt in configuration rather than scattered through the codebase. The abstraction has to cover what genuinely differs between providers: tool call formats, streaming event shapes, how the system prompt is passed, how stop reasons and refusals are reported. Wrap those, and swapping one provider for another, or a large model for a small one, becomes a config change plus an evaluation run instead of a refactor. Pinning matters as much. Floating aliases move underneath you, and a prompt tuned on one snapshot degrades quietly on the next, usually in the shape of the output rather than the substance, which is exactly the kind of break that slips past a smoke test and reaches a customer. Pin explicit versions, upgrade on purpose, keep the previous one reachable so a rollback is one deploy. That same abstraction is what later lets you route cheap work to a small model and hard work to an expensive one.
Cost control is an architecture decision, not a billing setting
Hosted models are billed per million tokens, with output usually priced several times higher than input, and reasoning tokens billed as output even though the user never sees them. Everyone checks those numbers. The bill is decided elsewhere: by how much context you send on every call, how many calls one user action triggers, and whether anything stops a loop. A chat that replays the whole transcript every turn pays for the same history again and again, so a long conversation costs roughly the square of its length, invisible in a three-message test. The failure we see most often is an agent that retries a failing tool, then retries the retry, and burns an afternoon of budget before anyone looks. Practical controls: cap input length before the call rather than apologising afterwards, send the easy majority of requests to a small model and keep the large one for cases that fail a cheap classifier, use prompt caching so the stable part of a system prompt bills at the cache read rate, not full price every turn, set a hard ceiling on tool-calling iterations, and give each feature a visible token budget. Caching has fine print: a minimum cacheable prefix, a short idle expiry, and any byte that changes earlier in the prompt (an injected timestamp, unsorted JSON keys) invalidates everything after it. Log token counts, model id and a feature tag on every request, or the invoice is one number and nobody can say which screen caused it. Handle 429s with jittered backoff and honour the retry-after header: retrying immediately at scale is how a rate limit becomes an outage.
Latency, streaming and the failures that only appear in production
A model call is slow by the standards of the rest of your stack, and the gap between acceptable and unusable is mostly perceived latency. Stream the response so the first token arrives fast, show what the system is doing while a tool call runs, and never leave a spinner with no state behind it. The infrastructure fights back here. Serverless functions have maximum durations that vary by platform and plan (Vercel's limits differ between Hobby, Pro and Enterprise at the time of writing), and a long generation will hit that ceiling. When it does, the honest answer is a job queue: persist a record, run the work somewhere long-lived, let the client poll or subscribe. Streaming also breaks in the middle boxes, because a proxy or CDN that buffers responses holds your tokens and delivers them in one lump, which looks to the user like the model being slow. Other things that only surface with real traffic: clients disconnect mid-stream, so persist partial output as it arrives rather than only at the end, otherwise a flaky mobile connection costs the user everything they typed. Network retries re-run side effects, so anything that writes or charges needs an idempotency key. Tool calling multiplies latency because every round trip is a full inference. Parallelise what is independent, and be honest with yourself about how many sequential model calls one screen can afford before people stop waiting.
Prompts are untested code until you build the evaluation set
Ask a team how they knew their last prompt change was an improvement and you learn most of what you need to know about the project. The answer should be a set of real inputs with expected properties, run automatically on every prompt edit and every model change. Assert on properties rather than exact strings: valid JSON against the schema, the correct SKU present, a refusal when the retrieved context does not contain the answer. Exact-match comparisons on generated prose fail constantly for no good reason, then get muted, which is worse than having no tests at all. Refusals need a designed path too. Models decline legitimate requests in security, medical, legal and moderation contexts, and a refusal rendered raw into a product UI reads as a bug. Detect it, fall back to a deterministic response or a human review queue, and log it. Data handling is a decision, not a default. Write down what never enters a prompt (card numbers, credentials, health records, another customer's data), redact before the call rather than after, and read the retention and training terms for the tier you are actually on, not the ones on the marketing page. Then keep a trace per request: model id, prompt version, token counts, latency, finish reason, validation result. When a customer forwards a bad answer three weeks later, that trace is the only thing that can explain it.
When a language model is the wrong tool, and when not to hire us
A language model is a probabilistic component. Anywhere the correct answer is defined by a rule, a deterministic system wins on cost, speed, testability and your ability to defend the result to a customer. Invoice totals, tax logic, shipping rules, permission checks, exact stock lookups, form validation, deduplication, sorting and scheduled reports should be code. We have replaced more than one AI feature with a lookup table and a regular expression: same input, same output every time, and a test suite that can prove it. Classification with a stable label set and enough historic examples is often better served by a small trained classifier than by prompting. If the underlying data is missing or contradictory, no model repairs that: retrieval on top of a bad knowledge base returns confident nonsense faster than a human ever could. We are also the wrong agency for some briefs. If you need on-site presence, meetings, video calls or someone sitting in your Slack all day, we do not work that way and you will find us frustrating. If you need model training from scratch, GPU cluster operations or research work, that is a different discipline. If you need round-the-clock on-call coverage, two people cannot honestly offer it. And if the goal is to have AI in the pitch deck rather than in the product, we will not be useful to you.
How two people in Dijon work with a client in California or Bangalore
We are a two-person agency in Dijon, France. Robin Monteiro builds (Next.js, React, TypeScript, the API and the data layer), Florian Loppion handles digital marketing. There is no account manager between you and the people writing the code, and no overseas office we could pretend to have. Everything runs in writing, by email, with a reply inside 24 business hours. No calls, no video, no scheduled slots. For an international client that removes the worst part of hiring abroad: nobody in California takes a 3am call, nobody in Bangalore waits until Tuesday for a slot in a French calendar. Work moves while you sleep and you read it when you wake up. Every decision, trade-off and rejected option also stays searchable months later, which matters more than it sounds when someone asks why an endpoint behaves the way it does. What changes on your side: write the brief instead of talking it through, batch your questions rather than firing them one at a time, and name one person who decides. Ambiguity a call would paper over has to be settled on the page. Our named references are French SMEs (Chouchou Ribeyre, Au Petit Detail, LB Athletic, Mediavocats, Vectosolve), so we will not have a case study from your market to show you. You own the code, the repository, the cloud accounts, the API keys and the domain name. Pricing is on request, quoted per project.
