Consensus
Consensus is the autonomous semantic layer that powers every Colrows runtime. It's a living, governed graph of business meaning - tables and joins, metrics, business terms, entities, events, and verified queries - that AI agents and analysts reason over before any data is touched.
This is the first of three chapters on the Colrows semantic layer. Here we cover what it is. Chapter 2, Semantic Pipelines, is how Colrows builds and maintains it. Chapter 3, the Catalog, is how you govern it.
Why analytics semantics aren't enough
Traditional semantic layers were built to support dashboards and BI tools. They focus on simplifying query construction for humans and standardizing metric definitions. AI agents are not analysts: they operate autonomously, continuously, and across domains. They need semantics that go far beyond metrics and dimensions - state, causality, constraints, event lifecycles, and organizational rules.
Pointing an analytics semantic layer at AI workloads produces brittle systems that work only for narrow query patterns and collapse under real-world complexity. Consensus is built differently: it is AI-native semantic infrastructure.
A machine-reasonable, governed semantic execution layer that lets AI agents reason, decide, and act safely over enterprise data - before any query is generated.
Semantic primitives
Consensus models meaning through typed, versioned assets. Together they define the semantic state space in which agents and humans operate.
| Primitive | What it represents | Examples |
|---|---|---|
| Table / Column | Source-aligned schema metadata plus semantic context - descriptions, profiles, and roles. | orders, orders.status. |
| Table Relationship | A governed join between two tables, with cardinality and optionality. | orders.customer_id → customers.id, many-to-one. |
| Category Set | The governed meaning of a coded column's values. | status = 'A' means Active. |
| Semantic Dataset | A reusable, governed row set that metrics build on. | "Inventory in system" - active, non-deleted inventory rows. |
| Metric | A governed measure: what to calculate, its grain, guidance, and usage examples - not a bare SQL fragment. | Net Revenue, Active Customers, Persistency. |
| Business Term | Business vocabulary and its canonical definition, with synonyms. | "An active customer is one who placed a non-refunded order in the last 90 days." |
| Business Entity | A core business noun. | Customer, Order, Subscription, Policy. |
| Event / Action | A business moment or state change, and what happens after it. | Order Placed, Signup, Cancellation, Claim. |
| Category / Relationship | A taxonomy label for grouping assets, and a link between two semantic assets. | "Regulatory", "Customer lifecycle"; Order triggers Refund. |
| Verified Query | Curator-approved, parameterized SQL for a known business question. | "Revenue for a region from a given date." |
Access and masking policies are not stored in the graph itself; they apply to the same tables and columns at query time. See Data access control.
Metrics as state, not queries
In Consensus, metrics are not stored SQL. They are modelled as derived semantic state - a continuously interpretable representation of business reality. A metric like Net Revenue does not merely encode how to compute a number; it encodes:
- Business meaning - what Net Revenue is, and how it differs from Gross Revenue or Bookings.
- Valid grain - per order, per invoice, per customer, per day.
- Dependencies - the exact Semantic Dataset versions and business terms it is built on.
- Guidance and examples - how it can be filtered, grouped, compared, and questions it does and does not answer.
- Downstream impact - which verified queries, cached SQL, and other assets depend on it.
When an agent observes a decline in Net Revenue, Consensus lets it reason semantically, not procedurally - distinguishing volume-driven decline from refund-driven erosion, because those relationships are explicit in the metric's state.
The semantic execution graph
All primitives connect through the semantic execution graph. It is not a passive knowledge store or a documentation artifact; it is an active runtime structure that agents consult while reasoning, deciding, and acting. Only current, valid versions are published to it, so runtime never reasons over a broken definition.
When an agent is asked to analyze a drop in Net Revenue, it does not simply join tables and aggregate. It follows semantic paths that link Net Revenue to its datasets, terms, and tables, and joins only along governed Table Relationships, each carrying its cardinality. Generated SQL is checked for safety, access, and cost before it runs.
Multi-vector embeddings
Each asset in Consensus is embedded as several vectors, not one:
- Label vectors - one for the asset's name and one for each synonym, so "lapse" and "churn", or "GMV" and "gross merch value", both find it.
- Description vectors - built from the governed description. A metric's also carries its grain, guidance, and usage examples; a verified query adds one for each alternate question.
Vectors identify candidates; every hit is then reloaded and checked against the current record, its validity, and the caller's access. Embeddings are never the source of truth.
Scopes & personas
Every asset belongs to your organization, and most are tied to the datasource they describe. On top of that, an asset has a semantic scope:
- Organization - the default. Definitions shared across the enterprise.
- Persona - a variant of an asset for one persona. When a request runs for that persona, the persona variant replaces the organization version of the same asset.
Persona variants are available for business terms today. Tables, columns, and table relationships are organization-scoped only, because they mirror the physical source.
Autonomous semantic agents
Consensus is operated by a coordinated set of automated components. They are not one-off workflow automations - they run over the semantic graph continuously to keep enterprise meaning accurate, consistent, and actionable. Semantic Pipelines covers each in detail.
- DiscoveryScanners and knowledge pipelines read schemas, profiles, query history, analytics platforms, and uploaded documents to propose tables, relationships, category sets, descriptions, and business terms.
- ValidationEvery asset is checked against the current catalog - datasets, tables, columns, join predicates, calculation SQL, grain. An asset that fails is marked invalid and kept out of runtime until it is repaired.
- LearningAnswered questions are captured as SQL resolutions in a semantic resolution cache, with references to the assets they used, so later questions can reuse proven SQL while it stays current.
- MonitoringChange records track every schema, profile, and definition change and drive dependent assets, projections, and cached SQL back into a consistent state.
Drift, conflict, and equivalence
Consensus runs continuous structural checks:
- Schema evolution - metadata scans compare a layout checksum, reconcile added, changed, and removed tables and columns, and record each change.
- Data drift - profile scans refresh missing or stale column profiles, which feed relationship discovery and category sets.
- Dependency-aware invalidation - a change that breaks a required input invalidates the dependent metrics, datasets, verified queries, and relationships, and marks cached SQL built on them stale.
- Conflicts - when definitions disagree or meaning cannot be inferred, Consensus raises an inquiry in the Learning Queue instead of guessing.
Governance & explainability
Every change to a node in Consensus is versioned, attributed (human or agent), timestamped, and linked to the agents, dashboards, and signals it impacts. This is the substrate the audit trail rests on. Every earlier version stays readable. An AI answer records the metrics, business terms, tables, and verified queries it used, with their versions, and an agent execution records the verified query version and the datasource's semantic revision - so you can see exactly which definitions produced a number.
Curators read and act on this versioning directly in Catalog - the workspace for browsing every node in the graph, reviewing agent-proposed changes, and inspecting the full version history behind any definition.
What Consensus enables
- An AI Analyst whose answers are repeatable, auditable, and explainable across teams and time (see AI Data Analyst).
- Agent-grade infrastructure - the same graph powers your custom agents over the API.
- Cross-system consistency - definitions remain stable across dashboards, signals, agents, and notebooks.
- Self-maintaining semantics - pipelines and change records keep the graph current, and curators only review what the pipelines cannot prove.