Bronze, Silver, and Gold are a great starting point, but real projects rename layers, split them, and stack them. Here’s how to decide what each layer is responsible for, and when a separate one earns its place.

Every CI/CD pipeline eventually needs something to run after deployment: initial setup, config syncs, validation checks. job_runs in Databricks Asset Bundles handles all of it natively, and you can use it to block a production update when validation fails.

Learn how to use Databricks Unity Catalog's governed tags to automatically flag undocumented or inactive tables, notify owners, and build a self-governing data catalog.

We benchmarked Databricks' new full-text search index across tables from 100M to 10B rows. The result: a 10B-row word search dropped from 71 minutes to under 11 seconds: a 99.75% speedup! Here's how the feature works, how to set it up, and when it's not worth using.

Migrating from BigQuery to Databricks is an organizational transition, not just a technical one. This retrospective covers the planning decisions, governance redesign, and cost realities that determine whether a migration delivers value or just progresses — drawn from real implementation experience.

Unity Catalog catalog commits move Delta table commit coordination from filesystem-level operations into Unity Catalog, making UC the authoritative source of table state. This enables strict concurrency control, attribute-based access for external engines, and multi-statement multi-table transactions — while leaving data in open Delta format on cloud storage.

Informatica's new LTS pricing means staying put is no longer free — it's a recurring tax on a platform with a shrinking capability roadmap. This guide covers the full migration path from PowerCenter to the Databricks Lakeflow stack: the CFO-ready business case, a construct-by-construct translation guide, and the five-phase M5 methodology SunnyData runs on every engagement. If your team is weighing IICS versus a full re-platform, the architecture decision and cost model are here.

Naming conventions only work if they're enforced — and a Confluence page nobody reads isn't enforcement. This post walks through using Databricks Workspace Skills to make naming rules executable in Genie Code, then scaling that to a catalog-wide audit agent built with Databricks Apps and DABS. The result is automated, repeatable governance that runs without requiring engineers to opt in.

Databricks Genie Code hit a 77.1% task success rate in production data science workflows — more than double what general-purpose coding agents achieve. But that performance is entirely conditional on the quality of your Unity Catalog metadata. SunnyData's two-week evaluation breaks down what works, what doesn't, and the governance layer you need before you go live.

Databricks' permanent Lakeflow Connect free tier delivers 100 DBUs per workspace per day — covering up to 100 million records of ingestion at no additional compute cost. For enterprise teams running multiple workspaces, that's over $255,000 in avoided annual costs. This post breaks down the economics, architecture, and what it means for teams still paying a third-party ETL tax.

Learn how temporary tables in Databricks SQL warehouses enable materialized data, DML operations, and session-scoped ETL workflows. Complete with practical examples.

Learn how Databricks' native Excel import handles multi-sheet workbooks, streaming autoloader, and SharePoint integration. Includes code examples.

Migrate external tables to managed tables in Databricks Unity Catalog using ALTER TABLE SET MANAGED while maintaining business continuity.

Master data relationships in Databricks Genie with and without foreign keys. Complete guide to using Unity Catalog constraints and defining custom joins for better AI insights.

Implement Databricks Unity Catalog access requests to streamline data permissions. Step-by-step guide covering BROWSE privileges, automated workflows, and ITSM integrations for better data governance.