Context window exhaustion mid-conversation
Session-scoped memory with selective compression, intent and key facts preserved without blowing token budgets.
← Back to Work
7UNIT / CASE STUDY 004
LLM assistants that survivereal operating constraints.
Most AI demos die in production. We owned the failure modes, memory, handoff, cost, tenant isolation, and shipped assistants that keep answering when conditions get hard.
THE SITUATION
Buyers were not asking for another chatbot demo. They needed assistants that would hold under real traffic: WhatsApp inquiry volume, multi-tenant cost pressure, and human escalation when the model is uncertain. Most LLM integrations fail the same way, they work in a demo and break in production.
THE STAKES
When an assistant fails at message 40, drops context on handoff, or burns budget without attribution, the business does not get a 'model problem', it gets silent leads, angry customers, and an AI line item nobody can explain. Production failure is a revenue and trust failure.
WHAT WE OWNED
7Unit owned architecture and delivery of production assistant systems across client contexts, including WhatsApp inquiry agents, designed around session memory, human handoff with conversation context, per-tenant cost attribution, and graceful degradation when the model layer fails.
Honest boundary
We do not claim a single named public client for this pattern page. The outcomes below describe production constraints we engineered for, not a single public scoreboard. We own the system constraints; the client owns domain policy and human staffing for escalations.
WHAT WE BUILT
LLM-powered assistant systems designed around the constraints that kill most implementations. Applied across WhatsApp inquiry agents, document intelligence pipelines, and coordination agents, same production discipline, different surfaces.
RISK REDUCED
Handoff that forces customers to repeat themselves
Human escalation carries full conversation context, last model reasoning state, and a suggested next action.
Unattributable AI cost across tenants
Every LLM call instrumented with tenant_id and conversation_id, cost anomalies visible before they become a surprise bill.
Hard failure when the model API is down
Graceful degradation: queue/retry for async work, immediate human handoff for live conversations.
ARCHITECTURE DECISIONS
Session memory without token blowout
Naive implementations keep full history in context for every call. We implement session-scoped memory with selective compression.
Rationale: A session that fails at message 40 because the context window is full is not a production system.
Trade-off accepted: Slightly more complex memory architecture in exchange for conversations that do not fail at scale.
Human handoff with full context transfer
When confidence drops, the system hands off with full context, the human does not start from scratch.
Rationale: A handoff that drops context is a restart, not an escalation.
Trade-off accepted: More complex handoff state management in exchange for seamless human escalation.
Cost attribution per tenant per conversation
Every LLM call carries tenant_id and conversation_id. Monthly cost reports per tenant are first-class.
Rationale: Without attribution, one high-volume tenant can make the system unprofitable without any single call looking wrong.
Trade-off accepted: Instrumentation overhead in exchange for full cost visibility per tenant.
Graceful degradation
When the LLM API fails, async work queues and retries; live conversations escalate to humans immediately.
Rationale: An AI system that fails hard when the model API is unavailable is not production-grade.
Trade-off accepted: More complex fallback logic in exchange for operational continuity under failure.
OUTCOMES
Outcomes
Designed-in
Failure modes
Memory, handoff, cost, and degradation treated as product requirements, not post-demo patches.
Per-tenant
Cost attributed
Token spend tracked per conversation and tenant, so AI cost is an operated line, not a mystery.
Context
Human handoff
Escalation carries the thread and suggested next action, not a cold restart for the customer.
STACK
Claude APIFastAPIPostgreSQLWhatsApp Business APIRedis
IF THIS SOUNDS LIKE YOUR PROBLEM
If your AI assistant works in a demo but you cannot explain cost, handoff, or what happens when the model fails, you do not have a production system yet. That is the problem we take ownership of.
Product and engineering leaders shipping customer-facing or multi-tenant LLM assistants who need production constraints designed in from day one.
CONTINUE EXPLORING