Token Efficiency Is a Data Strategy Problem, Not a Dev Problem

Most teams discover their AI cost problem the same way — on a billing dashboard, three months too late.

According to Monte Carlo’s Lior Gavish, who runs Claude across product, internal tooling, and agentic workflows at scale, teams routinely waste 30–40% of their token budget. Not through careless prompting, but through structural decisions made far upstream: bloated context windows, redundant data retrieval, and customer profiles stuffed with attributes that haven’t influenced a decision in six months.

For anyone building real-time customer engagement pipelines, this isn’t a developer problem to optimise away. It’s a data architecture problem — and it compounds.

Why Your Customer Data Layer Is the Real Token Budget

When a personalisation engine or conversational AI fires, it doesn’t pull from thin air. It pulls from whatever you’ve handed it — CRM fields, event history, segment tags, product catalogues. If your data layer hasn’t been designed with context relevance in mind, you’re essentially feeding a chef your entire pantry every time they need to make one dish.

The fix isn’t aggressive prompt compression. It’s deciding, at the data model level, what context is decision-relevant at the moment of engagement. Gavish’s framework at Monte Carlo distinguishes between what an AI needs to complete a task versus what happens to be available. That distinction — which most CEP implementations skip entirely — is where 30% of your token spend lives.

For a retention flow targeting lapsed users on Shopee or Lazada, the relevant context window is probably: last purchase category, days since last session, current promotion eligibility. Not their full 18-month event log.

Classical NLP Taught Us Something Token-Era Teams Keep Forgetting

There’s an instructive parallel in how classical NLP evolved. Research published on Towards Data Science tracing the arc from Bag-of-Words to stacked ensemble models on author identification tasks makes one thing clear: representation matters more than model complexity. TF-IDF with a well-tuned Naive Bayes classifier can outperform far more sophisticated architectures when the input signal is clean and the feature space is intentional.

The lesson transfers directly. LLMs don’t become more accurate when you throw more context at them — they become more expensive, and often noisier. The teams seeing the best output quality from Claude-based workflows, per Monte Carlo’s internal data, are the ones who treat context curation as a first-class engineering task: structured retrieval over full document dumps, recency-weighted signals over flat attribute lists, explicit task framing over open-ended prompts.

This is exactly what a well-designed customer data platform should be doing before the AI layer ever sees a request.

---

The CEP Angle: Batch Thinking Breaks Real-Time AI

Here’s where engagement architects need to be honest about their own stack. Most CEPs — even nominally “real-time” ones — were designed around batch-logic assumptions. Profiles are refreshed on a schedule. Segments are computed overnight. The event stream feeds a warehouse that feeds a downstream job.

When you bolt an LLM onto that architecture, the token waste problem gets worse, not better. You’re not passing the AI a live signal — you’re passing it a snapshot that may be hours old, padded with stale attributes, formatted for human readability rather than machine efficiency.

Building for real-time, context-aware engagement means rethinking the handoff. What does the AI actually need at the moment a user opens a LINE message or taps into a Grab rewards screen? That moment-specific context window — assembled dynamically from fresh signals, not pre-computed profiles — is the architectural shift that makes token efficiency a strategic lever rather than a cost-cutting afterthought.

Teams running on GCP’s Vertex AI or AWS Bedrock in Southeast Asia are already finding that streaming event architectures (Kafka-backed, with lightweight feature stores) reduce context payload size by 40–60% compared to passing full CRM exports — with measurably better personalisation accuracy because the signal is cleaner.

Implementation: Where to Start Without Rebuilding Everything

You don’t need to rearchitect your entire data stack. Three practical starting points:

Audit your context payloads. Log what you’re actually passing to AI-assisted flows today. In most implementations, 50–60% of attributes passed are either never referenced by the model or could be computed on-demand rather than pre-loaded. This audit typically takes a day and surfaces immediate wins.

Introduce a relevance gate. Before any customer context enters an AI workflow, run it through a lightweight scoring layer — either rules-based (recency + engagement signal weight) or a small classification model. Monte Carlo’s approach uses structured metadata schemas to pre-qualify what goes into the context window. The result is smaller payloads and more consistent outputs.

Align your data team and your engagement team on the same token budget conversation. This is the organisational fix. Token efficiency stops being a dev concern and starts being a strategy lever the moment data architects and journey designers are in the same room, arguing about what context actually matters at each touchpoint.

---

The brands winning at AI-powered engagement in Southeast Asia won’t necessarily have the biggest models or the most sophisticated prompts. They’ll be the ones who treated their data layer as the product — designing clean, relevant, moment-specific context as a core competency rather than an afterthought.

As AI costs stay volatile and real-time personalisation expectations rise across the region’s mobile-first markets, the question worth sitting with is this: if your data architecture was designed for batch reporting, what would it actually take to rebuild it for the latency and precision that real-time engagement demands?

---

At grzzly, we help growth and data teams across Southeast Asia design CEP architectures that are built for real-time from the ground up — not patched together from batch-era infrastructure. If you’re hitting token cost ceilings or finding that your “personalisation” is lagging reality by hours, that’s a data architecture conversation, not a prompt engineering one. Let’s talk