The setup
One of our clinic deployments runs patient intake on a messaging channel. Patients write in natural language — reschedule requests, treatment questions, “can I come Thursday afternoon instead” — and an AI assistant handles the conversation. The clinic’s schedule lives where the clinic already keeps it: a shared spreadsheet grid the front desk maintains, with slot conventions the staff have used for years.
The obvious build is to let the model read the schedule and book whatever looks open. We did not build that, and the reasons are the useful part of this post.
Rule one: the model never decides what is bookable
The clinic’s schedule encodes policy in conventions: certain slots are reserved for follow-up visits and consultations only, and they are visually marked, not labeled in a database. New-patient treatments do not belong in them, ever. The front desk knows this. A language model reading the grid does not — it sees an open slot.
So the first thing we shipped was not booking at all. It was a hard gate: reserved slots are structurally off-limits to the assistant. Not “instructed to avoid” — off-limits, enforced in code that runs after the model and before anything touches the schedule. A prompt can be argued with; a pure function cannot.
That is the architecture in one sentence: the LLM extracts, deterministic code decides. The model is excellent at reading “next Thursday after 3 would be great, same treatment as last time” and producing structured intent: requested window, treatment type, patient identity. Everything after that point is plain, testable logic — slot classification, eligibility rules, duration math — written as pure functions over data, with unit tests, reviewed like any other code.
The boring details carry the safety
Most of the engineering effort went into details no demo would ever show:
- Treatment durations come from the clinic’s own reference table. Every bookable treatment has a preparation time and a treatment time, reconciled line by line against the clinic’s printed table — including the pass where we found and fixed nineteen mismatches. The assistant cannot offer a slot the real-world duration does not fit.
- Check-in time is derived, not guessed. Preparation time determines when the patient should arrive; the confirmation message states it. One less judgment call for the model, one less surprise for the front desk.
- Phone numbers are written as text. A spreadsheet will happily eat the leading zero off a phone number stored as a number. Small bug, real-world consequence, deterministic fix.
- Ambiguity routes to a human. When the request does not fit the gates cleanly — an unusual treatment combination, a slot conflict the rules cannot resolve, a patient asking for something the policy table does not cover — the assistant does not improvise. It hands the conversation to a person, with the context attached. The handoff is a feature, not a failure mode.
Each decision rule in that list exists because the clinic’s operator made a call, we encoded the call, and the encoding is now inspectable. When the operator changes a policy, we change a function and its test — not a prompt and a hope.
Why we work this way
There is a version of this project that ships in a weekend: give a capable model the schedule and a system prompt, let it book. It would demo beautifully and work most of the time. “Most of the time” is the problem. A clinic schedule is a promise to real patients and real staff; the failure cases are double-booked treatment rooms and a front desk that stops trusting the tool after the second mess.
The split we use — model for understanding, deterministic gates for authority, human handoff for everything outside the gates — costs more engineering up front and gives back something a prompt cannot: the booking logic is testable before it touches a patient, auditable after, and correctable in exactly one place when policy changes.
The pattern is not specific to clinics. Anywhere an AI assistant meets a resource that other people depend on — a calendar, an inventory, a ledger — the same line applies. Let the model read. Do not let it rule.