What's new in Colrows.

Governed agent access, documents & unified analytics

Core upgrade: The Colrows Model Context Protocol (MCP) implementation ships as the supported surface for agent access to the semantic layer — REST endpoints, OAuth discovery, telemetry, and tool-call continuation. Alongside it, a durable document-processing pipeline and a unified AI Analyst data cache extend governed execution to unstructured sources and to repeat queries.

Model Context Protocol

• Colrows MCP server with REST endpoints, OAuth discovery, tool-call continuation, and the ask_metadata endpoint.
• MCP tools for metadata inspection, column profiles, join finding, and governed AI data access.
• OAuth compatibility, cursor and continuation handling, and durable authorization/grant persistence. MCP is now the supported integration surface; cluster-based CLI clients were retired.

Documents & conversations

• Document upload and a durable processing pipeline with S3 storage, multipart-upload fixes, chunk retry, and generation IDs.
• Conversation processing and conversation-signal extraction now feed proposal generation and semantic-asset promotion.

Datasets, queries & dashboards

• Unified AI Analyst data cache with concurrent, lock-guarded dataset loading, configurable eviction/refresh, and a cache-policy UI.
• SQL fingerprinting (deep-chain safe, select-column-order-insensitive) scoped to the SQL editor; dashboard widget dataset refresh.
• analyze_data renamed to ask_data, with dynamic continuation and clarification handling for AI responses.

Experience & administration

• The former Workspace merged into the SQL Editor; token-usage, conversation, inquiries, and cache-policy views added.
• AWS Marketplace onboarding, account-based organization identity and metering, and unconditional Cloudflare Turnstile; Google and Microsoft SSO hardened.
• Retired the Notebook, Join, Category Set, Schema, subscription, and History screens as their concepts consolidated.

Upgrade notes

• AI Data API: update MCP and tool callers that referenced analyze_data to ask_data.
• Join discovery: tool/API names and response models changed — regenerate clients or update response adapters.
• Models: document request/event models and chunk identity changed; signup is now UserRegistration and ViewState is now ViewRevisions.
• Organization CLI: no database is created automatically — add an explicit datasource/database step. Remove bookmarks or integrations that target retired routes.

Semantic platform re-architecture

Core upgrade: The semantic core was rebuilt for durability and scale. Concept lookup moved to a single MongoDB Atlas vector collection, the consensus pipeline became a durable scanner / capability / change-consumer control plane, and a new Semantic Signal → Claim → Proposal → Asset workflow governs how meaning is generated, promoted, and versioned — every stage bounded by an immutable semantic scope.

Semantic search & asset generation

• Unified MongoDB-backed vector collection with Atlas search indexes, replacing entity-specific vector models; semantic search now spans actions, events, business entities, and semantic categories.
• Signal → Claim → Proposal → Asset pipeline with proposal generation and refinement, promotion triggers, token-consumption metadata, and immutable SemanticScope boundaries on every stage.
• The Business Term Anchor model replaced by TableHint; added semantic associations, confidence values, grain models, and support for views and materialized views.

Consensus control plane

• Durable consensus service built on build sessions, source runs, task runs, schedules, and independently processed change consumers, with onboarding and periodic execution modes.
• Source scanners for column profiles, query history, catalog metadata, and join discovery, plus a durable ChangeRecord lifecycle with lineage, causation, and synchronous vector projection.

Data sources & catalogs

• New catalog connection framework with refresh policies and a manual-refresh endpoint; added MindsDB and Dremio onboarding and Databricks information_schema layout extraction.
• Query-history extraction, column-profile scanning, foreign-key-to-join conversion, and join discovery; custom datasource authentication and Azure Key Vault secret management.

Storage, reliability & operations

• Distributed-lock lifecycle hardening, a MongoDB store-manager CLI with index generation and migration utilities, and configuration consolidated into colrows.yaml.
• AWS development deployment workflow, secret-backed certificate handling, Prowler AWS security scanning, and Dependabot-driven upgrades across security- and runtime-sensitive dependencies.

Upgrade notes

• Scoped pipeline: legacy signal/claim/proposal runs without a scope must be recreated; pre-scope records are rebuilt rather than treated as organization-scoped.
• Business terms: Anchor is replaced by TableHint and a tableId is added — update serialized models and response parsing.
• Datasource drivers: bundled MySQL and Snowflake drivers were removed — supply and configure the appropriate driver artifacts explicitly.

Multi-vector embeddings & expanded dialect support

Core upgrade: Semantic concept lookup now operates on multi-vector embeddings per entity (definition, usage, combined) instead of single-vector similarity. Dialect compilation extends to ClickHouse and Trino federated query engines, maintaining determinism across a wider ecosystem.

Performance & compiler optimization

• Compile-time policy evaluation: up to 3× faster on graphs with 10K+ entities through optimized constraint resolution.

Fixes & stability

• Resolved edge case in join path proof when a metric crosses 4+ datasets with ambiguous cardinality.

Compile-time governance, GA

Core upgrade: Governance is no longer a runtime filter bolted onto query execution. RBAC and ABAC policies are now materialized into the execution plan during compilation. A persona's allowed subgraph is resolved before any SQL is generated, unauthorized queries fail compilation (loud, safe, auditable), and no warehouse resources are spent on queries that should never have run.

Governance & security

• Row and column-level predicates compiled per persona.
• Point-in-time reproducible audit log: re-run any historical query with the exact definitions and policies in force.

Fixes & stability

• Query-explosion guards: ambiguous joins now fail compilation with clear, actionable error messages.

Conversational AI Analyst

Core upgrade: Natural-language questions now compile through the semantic graph into deterministic SQL with full governance applied. Every answer ships with its reasoning chain (entities resolved, metrics matched, join paths proven, filters applied, policies evaluated), not a probabilistic guess. Slack integration brings governed analysis into team workflows.

Performance & compiler optimization

• Stateful conversational scope: follow-ups resolve against the same graph version and policy context, eliminating semantic drift mid-conversation.

Semantic Studio

Core upgrade: A visual editor for the semantic graph brings definition, relationship mapping, and governance rule authoring into a single plane. Multi-scope architecture (global to datastore to persona to user) allows semantic inheritance and override at every level, enabling both centralized governance and localized customization.

Performance & compiler optimization

• Autonomous maintenance: drift detection and conflict resolution run continuously across thousands of entities, keeping the semantic graph current without manual intervention.
• Versioned graph with structural diffing: every change is tracked, enabling rollback and point-in-time reproducibility.

16+ data sources & dialect-perfect SQL

Core upgrade: Support for Snowflake, Databricks, BigQuery, Redshift, Postgres, MySQL, and 10+ additional data sources. The compiler produces dialect-perfect SQL for each engine (window functions, CTEs, aggregation syntax, quoting rules) while guaranteeing deterministic semantics across all backends. No data movement required.

Performance & compiler optimization

• Inline visualization and query-plan introspection: inspect compiled SQL, join cardinality, and estimated costs before execution.

Colrows Semantic Execution Layer - GA

Core upgrade: The Colrows semantic execution runtime launches with four computational domains: intent parsing, semantic resolution, constrained planning, and governed execution. Each stage is isolated to ensure determinism, reproducibility, and independent scalability. The foundation is a typed, multi-scope, versioned knowledge graph encoding entities, metrics, relationships, policies, personas, and scopes as first-class graph objects. The initial AI Data Analyst Agent consumes this graph, compiling natural-language questions into deterministic, auditable SQL.