A friend recently shared a build log for a data agent at a Series C startup. Three weeks of work. One engineer. The result: an agent that answers data questions in Slack by investigating the codebase on demand, writing SQL, executing it, and delivering the answer. It replaced a planned hire of four to five data analysts.
The architectural insight behind it is worth paying attention to: the semantic layer is dead. Context management killed it.
For years, the conventional wisdom in data engineering was to build a semantic layer — a static, hand-maintained mapping between business concepts and SQL. Define every metric. Map every dimension. Maintain it religiously. It was the backbone of the “modern data stack.” And it was a pain in the ass.
The new approach inverts this entirely. Before writing any SQL, a sub-agent — a lightweight investigative process — reads the actual transformation logic in the data pipelines, traces dependencies upstream, validates business rules against application code, and builds a brief for itself: relevant tables, column definitions, join paths, filters, dedup logic, caveats. The semantic layer, rebuilt dynamically at query time. No maintenance required.
The reason this works now and didn’t work two years ago is model capability. With Opus 4.6, the need for a hand-maintained semantic layer has collapsed. The model can read raw schema, raw transformation code, and raw business logic, and synthesize the mapping on the fly. The semantic layer was always a crutch for models that couldn’t reason well enough to figure it out themselves.
I read this and immediately saw credit unions. Because every problem this solves — siloed data, institutional knowledge locked in people’s heads, questions that change faster than dashboards can keep up — is the credit union data problem magnified ten times over.
The Architecture That Makes This Work
The startup build has four components worth stealing. I’ll describe each one, then explain what changes when you bring it to a regulated financial institution.
1. The Context Agent. Before the main agent writes any SQL, a sub-agent investigates the data landscape. It reads the actual transformation logic — not documentation, not a schema diagram, the code itself. It traces dependencies. It checks for known issues. It produces a structured brief: these are the tables you need, these are the join paths, these are the gotchas. This is cheap to run (it’s mostly file reads) and it’s what prevents the main agent from hallucinating table structures.
In a startup, the context agent reads dbt models and application code. In a credit union, it reads the normalized data layer that sits between your core processor and the agent. Same pattern, different source material. The context agent doesn’t care whether it’s reading a dbt model or a Symitar-to-normalized mapping — it just needs access to the truth about how data flows.
2. The Self-Correcting Knowledge Store. When a user corrects the agent — “that’s wrong, dealer XYZ is indirect but you’re showing them as direct” — the system extracts the correction into a durable, reusable piece of knowledge. The startup engineer calls them “quirks.” Short, specific, retrievable: “The orders table has duplicate rows per order when there are multiple shipments; always dedup on order_id.”
These quirks get stored with embeddings for semantic search. On each new question, the system runs hybrid retrieval — vector similarity plus keyword search — and injects the top matches alongside the context agent’s brief. Over time, this becomes your institutional knowledge base.
For credit unions, the quirks concept is transformative. Every credit union has knowledge like this that lives exclusively in someone’s head: “The indirect channel codes changed in October.” “Participation loans show up as duplicate records.” “The ACH file from First National uses a non-standard format.” Today, when that person retires, the knowledge walks out the door. In a quirks store, it compounds.
3. Human-Authored Metric Definitions. For your core KPIs — the metrics that get asked about constantly and have precise definitions — let your team author structured definitions with inference guidance. How to calculate it. What filters apply. What the NCUA Call Report definition requires versus what your internal definition uses. These go into the same knowledge store as quirks and get retrieved the same way.
Think of this as the 20% of the semantic layer that was actually useful, maintained by humans when they feel like it rather than required for the system to function. The other 80% — the exhaustive mapping of every possible dimension and metric — is what the context agent handles dynamically.
4. The Self-Scoring Recovery Loop. This is the part most people skip, and it’s what separates a demo from a production system. The agent scores its own SQL on three dimensions: structural correctness, execution reliability, and alignment with the original question. It breaks the question into sub-questions and checks whether each one is actually answered by the query.
If the score is low, it doesn’t just retry blindly. It builds a context-gap brief — what’s missing, what’s misaligned — and either reruns context enrichment or retries with a targeted prompt focused on the specific gap. This is the difference between an agent that confidently delivers wrong answers and one that catches its own mistakes.
What Changes in Regulated Financial Services
The startup version of this is elegant. The credit union version needs three additional layers that generic data tools will never have.
Compliance-aware context enrichment. When the context agent investigates a question, it doesn’t just map tables and joins. It checks: does this query touch protected class data? Does this analysis need to match a regulatory methodology? Should this result set be filtered for BSA/AML restrictions or opt-out preferences? The compliance check happens at the investigation stage, before any SQL is written — not as a filter applied after the fact.
The quirks store in a credit union isn’t just “this table has duplicates.” It’s “this metric must match the Call Report definition” and “this analysis requires fair lending review before distribution” and “this member segment has enhanced due diligence requirements.” Compliance knowledge lives alongside data knowledge in the same retrieval system.
Cross-system normalization. The startup has one data warehouse. A credit union has three to five systems with fundamentally different data models, batch processing cycles, and integration capabilities. The context agent needs a normalized layer to investigate — you can’t point it at raw Symitar PowerOn output and expect coherent results.
This is the hard engineering work. CDC pipelines that capture changes across systems. Data normalization that reconciles different schemas into a queryable model. This is the part that takes months, not weeks, and it’s the part that no amount of prompt engineering can substitute for.
Audit trails. Every query the agent runs, every correction a user makes, every quirk that gets stored — all of it needs to be logged, traceable, and examinable. When your NCUA examiner asks how a particular number was generated, you need to show the exact query, the exact data it touched, the exact context brief the agent used, and the exact knowledge store entries that influenced the result. The startup doesn’t need this. You do.
The Economics
The startup comparison is useful for framing the economics.
A data warehouse implementation for a credit union — Snowflake or BigQuery, ingestion layer, dbt, BI tool — runs $200,000 to $500,000 in year one, plus $100,000 to $200,000 annually. And the output is dashboards. Static answers to questions someone thought to ask six months ago.
The startup built a data agent in three weeks with one engineer that answers any question on demand and gets smarter over time. The infrastructure cost is a database, an embedding model, and API calls to a language model. The ongoing cost scales with usage, not with maintenance headcount.
The credit union version takes longer than three weeks because of the normalization and compliance layers. But the operating cost model is the same: infrastructure plus API calls, not headcount. And the output isn’t dashboards — it’s answers. Any question, any time, from anyone authorized to ask.
Your CFO stops building board reports in Excel. Your lending VP stops waiting three days for a delinquency breakdown. Your compliance officer stops manually cross-referencing systems for exam prep. Not because you hired more analysts, but because you built a system that does what analysts do — investigate the data, apply institutional knowledge, deliver answers — and made it available to everyone.
Why This Matters Now
Two things changed in the last six months that make this architecture viable where it wasn’t before.
First, the models got good enough. Opus 4.6 can reason over raw schema and business logic well enough to replace a hand-maintained semantic layer. This was not true with GPT-4o. It was marginally true with the first Opus. It’s decisively true now.
Second, the self-correcting knowledge store pattern matured. The combination of vector similarity search and keyword search for hybrid retrieval, stored corrections as durable quirks, and human-authored metric definitions as a lightweight semantic overlay — this pattern has been battle-tested at production scale in multiple environments. It works.
Credit unions are sitting on decades of member data locked in systems that can’t talk to each other. The semantic layer approach to unlocking that data was too expensive, too brittle, and too slow. Context management — agents that investigate on demand, learn from corrections, and carry compliance awareness in their bones — is the architecture that finally makes credit union data accessible to the people who need it.
The semantic layer is dead. What replaces it is better.
