An Agentic Data Platform Meets Databricksʼ AI Coding Agent
Auraa and Genie Code
Better Together - An Agentic Data Platform Meets Databricksʼ AI Coding Agent
Genie Code writes SQL and Python, builds Lakeflow pipelines, debugs failures, and monitors production workloads. For individual data practitioners, it changes how work gets done. But an organization's data infrastructure runs on governance, multi-tenant isolation, audit trails, and reproducible pipelines and those are platform responsibilities, not agent responsibilities. Genie Code was not built to carry them.
This whitepaper documents how Auraa picks up exactly where Genie Code stops.
Together, they cover the full picture: Genie Code handles the code, Auraa handles the configuration, governance, and organizational context that makes that code trustworthy at scale.
Download the White Paper to Learn:
- The curation tax: why catalog quality directly determines how well Genie Code performs and how Auraa eliminates the manual curation burden structurally at provisioning time
- Configuration vs. code generation: why metadata-driven pipeline specs stored in Delta Lake give compliance teams the audit trail and reproducibility that generated code alone cannot provide
- The three-meta-tool pattern: how Auraa exposes 166 platform operations to Genie Code in just 3 MCP tool slots, with built-in discovery and full authorization on every invocation
- Workflow prompts and domain skills: how Auraa transfers platform knowledge and data engineering best practices to any MCP-compatible agent without consuming tool budget
- Active governance in practice: grant policy registries, drift detection across tenant catalogs, and the reconciliation workflow that keeps permissions aligned with declared policy
Frequently Asked Questions
What is the curation tax and how does Auraa eliminate it for Genie Code?
The curation tax is the performance penalty Genie Code pays when the Unity Catalog metadata it relies on for context is incomplete, stale, or inconsistently documented. Genie Code uses catalog metadata (table descriptions, column annotations, relationship documentation) to understand what data exists and what it means. When that metadata is missing or wrong, the quality of generated code degrades proportionally. Auraa eliminates the curation tax structurally at provisioning time: when a new tenant or data source is onboarded, Auraa's agents populate the required catalog metadata as part of the provisioning workflow, rather than leaving it as a manual task for data engineers.
Why does metadata-driven configuration give compliance teams what generated code alone cannot?
Generated code (SQL, Python, Spark) is an output. When compliance teams need to audit a pipeline, they must read and interpret the code to understand what transformation was applied, under what quality rules, to which tenant's data. Metadata-driven pipeline specs stored in Delta Lake express those decisions as structured, versioned records: what source, what transformation contract, which quality thresholds, which tenant scope. The audit trail is the metadata itself, not a log of code executions. This gives compliance teams a reproducible, human-readable record without requiring them to interpret generated code.
What is the three-meta-tool pattern and how does it expose 166 platform operations in 3 MCP tool slots?
MCP (Model Context Protocol) agents like Genie Code consume tool slots from a fixed budget per session. Exposing 166 platform operations as 166 individual MCP tools would exhaust that budget immediately. The three-meta-tool pattern packages all 166 operations into 3 meta-tools: one that discovers which operations exist and what they require, one that invokes a named operation, and one that checks authorization before invocation. Genie Code uses the discovery tool to find the operation it needs, confirms authorization, then invokes it -- accessing the full Auraa platform capability without consuming tool budget on individual operation definitions.
What are workflow prompts and domain skills in Auraa's integration with Genie Code?
Workflow prompts and domain skills are structured knowledge packages that Auraa delivers to any MCP-compatible agent, including Genie Code. Workflow prompts encode Auraa's platform knowledge as reusable prompt templates -- for example, the correct sequence of operations to onboard a new data source, or the standard quality validation workflow for a silver layer promotion. Domain skills encode data engineering best practices specific to Auraa's architecture. Both transfer platform knowledge to the agent without consuming MCP tool budget, because they are delivered as prompt context rather than as tool definitions.
What is active governance in Auraa and how does drift detection work?
Active governance means Auraa continuously verifies that the permissions actually applied in Unity Catalog match the policy declared in Auraa's grant policy registries. Drift detection runs across tenant catalogs and flags any divergence: a permission that was granted outside Auraa's provisioning workflow, a catalog structure that does not match the declared isolation strategy, or a Unity Catalog role that has accumulated permissions beyond its declared scope. The reconciliation workflow then identifies the corrective actions needed to realign actual permissions with declared policy.