Customer memory at the level of a 20-year veteran (without hiring one)

There is a specific kind of person every long-running small business has on payroll: the 20-year-veteran receptionist. The one who knows every customer's name, their kids' names, what unit Tom installed in the spring, which person in the family is allergic to peanuts, and whether the dad is the easy one or the hard one to deal with on the phone. That person is worth more than they make. They are also, for most service businesses, irreplaceable: when they retire, the institutional memory leaves with them.

This post is about how the AI agent reproduces what they did. Not with a ChatGPT-flavoured CRM that happens to have an LLM glued to the side, but with durable customer memory that grows from real interactions, surfaces relevant facts at the right moment, and actually changes the experience for repeat callers. We will walk through what it remembers, how it uses what it remembers, and where it goes wrong.

What gets remembered

Three categories, all built automatically from interactions:

The third category is the one most CRMs do badly. A traditional CRM has fields: name, phone, email, last visit. That is fine for compliance and reporting; it is not "customer memory" in any meaningful sense. The interesting thing about a 20-year veteran is they remember which customer is sensitive about price, which one tips well, which one always asks "is this covered," which one will recommend you to their neighbour if you do a small extra thing. None of those facts go in a CRM dropdown.

The agent's customer memory captures those facts as free-form notes attached to the customer record. Each note is timestamped and cited (which call, which transcript line, which message). When the customer calls again, the relevant notes are loaded into the agent's context for the call.

What it changes about the experience

The visible behaviour of an agent with serious customer memory is small but cumulative.

Greeting by name. When a known number calls, the agent picks up with "Hi Mrs. Henderson." The customer notices. They feel known.

Skipping known questions. The agent does not ask for the address it already has, does not ask which unit they have, does not re-confirm the warranty status. The conversation feels human-paced.

Calling out anomalies. "I see Tom installed your AC in February. Is this about that, or something different?" The agent connects the new call to the relevant history without being told.

Honouring rejections. "I won't offer you the protection plan again; you said last time you weren't interested." The agent does not nag.

Surfacing context for handoff. When the agent escalates to the owner, the handoff includes "Mrs. Henderson, repeat customer, 7 visits, last service was the AC tune-up in March. She mentioned a thermostat issue last fall that wasn't covered, may need attention next visit." The owner picks up the phone with a complete brief.

How the memory grows

Two paths.

Conversational. Every call, every text, every WhatsApp turn becomes notes against the customer record. The agent decides what is worth keeping (specifics, preferences, complaints, important facts) and what to discard (chitchat, fillers). A regular consolidation pass promotes notes that have been mentioned more than once into stable beliefs.

Structured. When the agent does a job, the job's metadata (date, service, technician, parts, payment) becomes structured fields on the customer record. These are queryable: "show me every customer Mark serviced in March" returns rows.

The two paths complement each other. The structured fields answer "what happened." The conversational notes answer "what was said about it."

What you can ask

A representative set of useful queries against customer memory:

You: what did the Henderson family have done last time
Ajeris: Mrs. Henderson, AC tune-up, April 14th. Mark serviced.
        $320, paid same day. Mark noted her thermostat is older
        and flagged it for next visit.
 
You: who hasn't been seen in over a year
Ajeris: 14 customers. The 5 with the highest historical revenue:
        Patel ($3.2k LTV, last seen Feb 2025), Okonkwo ($2.8k, Jan 2025)...
 
You: any callbacks I owe
Ajeris: Two. Mr. Reyes asked for a callback Tuesday about the
        capacitor question. Mrs. Lopez called yesterday and said
        she'd think about the install quote.

This is the surface that converts a customer-list-with-fields into something that behaves like the 20-year-veteran's brain. You ask in plain English. The agent answers from memory.

Where it goes wrong

Three failure modes.

Stale memory. The agent remembers Mrs. Henderson preferred Tuesdays in 2024, but she has changed jobs and Tuesdays no longer work. If she has not told the agent, the suggestion will be wrong. The fix is mid-call correction: "actually, Tuesdays are bad now," and the agent updates the preference. After one correction, the memory is current.

Mistaken identity. The customer is calling from their spouse's phone, the agent assumes it is the spouse. This happens about 5% of the time on shared-household numbers. The fix: the agent identifies who it is talking to in the first sentence ("Hi, am I speaking to Mrs. Henderson?") and corrects the assumption when wrong.

Memory drift. The agent's interpretation of an old note has shifted; the original note said "kitchen sink leak" but the consolidation drift labelled it "plumbing issue." The fix: when the agent says something specific about history, it cites the source. You can ask "where did I say that?" and the agent shows the original note.

Privacy, deletion, export

The customer is the data subject. Three controls:

• Right to know. Customers can request a full export of what is stored about them. The agent produces a JSON dump or a human-readable summary.
• Right to delete. Customers can request deletion. The agent honours it and confirms.
• Granular rejection. Customers can ask "do not store this" mid-conversation; the agent flags the conversation as ephemeral, no notes are written.

These are operational defaults. They are also not negotiable; small-business compliance with consumer-data norms is non-optional in most US states by 2026.

Try it

Call your own number. After your second call (with a different scenario), notice whether the agent recognises you. The recognition is the test.