Migrating from BigQuery to Databricks is an organizational transition, not just a technical one. This retrospective covers the planning decisions, governance redesign, and cost realities that determine whether a migration delivers value or just progresses — drawn from real implementation experience.

Unity Catalog catalog commits move Delta table commit coordination from filesystem-level operations into Unity Catalog, making UC the authoritative source of table state. This enables strict concurrency control, attribute-based access for external engines, and multi-statement multi-table transactions — while leaving data in open Delta format on cloud storage.

Informatica's new LTS pricing means staying put is no longer free — it's a recurring tax on a platform with a shrinking capability roadmap. This guide covers the full migration path from PowerCenter to the Databricks Lakeflow stack: the CFO-ready business case, a construct-by-construct translation guide, and the five-phase M5 methodology SunnyData runs on every engagement. If your team is weighing IICS versus a full re-platform, the architecture decision and cost model are here.

Naming conventions only work if they're enforced — and a Confluence page nobody reads isn't enforcement. This post walks through using Databricks Workspace Skills to make naming rules executable in Genie Code, then scaling that to a catalog-wide audit agent built with Databricks Apps and DABS. The result is automated, repeatable governance that runs without requiring engineers to opt in.

Databricks Genie Code hit a 77.1% task success rate in production data science workflows — more than double what general-purpose coding agents achieve. But that performance is entirely conditional on the quality of your Unity Catalog metadata. SunnyData's two-week evaluation breaks down what works, what doesn't, and the governance layer you need before you go live.

Databricks' permanent Lakeflow Connect free tier delivers 100 DBUs per workspace per day — covering up to 100 million records of ingestion at no additional compute cost. For enterprise teams running multiple workspaces, that's over $255,000 in avoided annual costs. This post breaks down the economics, architecture, and what it means for teams still paying a third-party ETL tax.

Learn how temporary tables in Databricks SQL warehouses enable materialized data, DML operations, and session-scoped ETL workflows. Complete with practical examples.

Learn how Databricks' native Excel import handles multi-sheet workbooks, streaming autoloader, and SharePoint integration. Includes code examples.

Migrate external tables to managed tables in Databricks Unity Catalog using ALTER TABLE SET MANAGED while maintaining business continuity.

Master data relationships in Databricks Genie with and without foreign keys. Complete guide to using Unity Catalog constraints and defining custom joins for better AI insights.

Implement Databricks Unity Catalog access requests to streamline data permissions. Step-by-step guide covering BROWSE privileges, automated workflows, and ITSM integrations for better data governance.

Implement granular secret access control in Unity Catalog that goes beyond traditional Key Vault-level permissions. This advanced approach uses Unity Catalog UDFs with service credentials to create secret-specific access controls, allowing you to grant users access to individual secrets rather than entire Key Vaults.

Databricks Lakehouse excels at analytical workloads but struggles with single-row lookups that customer-facing applications demand. This blog shows how to solve the OLAP vs OLTP dilemma by adding PostgreSQL-based OLTP capabilities to your existing Lakehouse architecture

Enhance your Unity Catalog lineage by incorporating external data sources such as Kafka streams and IoT devices. This blog covers both UI-based and programmatic methods for creating complete data lineage visibility in your Databricks environment.