A hands-on preview of Genie ZeroOps, Databricks' autonomous agent for monitoring and fixing data pipeline issues — tested against a data overflow bug, a division-by-zero error, and a live migration.

Databricks Asset Bundles direct mode is now GA, and it's already the default for bundles created from the workspace, with full CLI default rollout on August 26, 2026. By removing the Terraform dependency, direct mode cuts deployment time by up to 40%, replaces opaque state with human-readable JSON, and unlocks resources like instance pools and Genie Agents that Terraform-gated releases used to delay.

Does 'streaming means Kafka' still hold for single-sink ingestion into Delta? We benchmarked it with 64.8 million real events and found a cost gap that never closes, 3.6× to 20.9×

Most teams assume faster real-time queries mean higher compute costs. We benchmarked Databricks' new RT Lakehouse engine across datasets from 1 million to 50 billion records to find out, and the results reset what "fast" costs on a lakehouse.

Databricks just replaced the engine behind DABs deployment. Here's what Direct Engine changes, why it matters, and how to migrate your existing bundles before Terraform support goes away.

We benchmarked Databricks' new full-text search index across tables from 100M to 10B rows. The result: a 10B-row word search dropped from 71 minutes to under 11 seconds: a 99.75% speedup! Here's how the feature works, how to set it up, and when it's not worth using.

Migrating from BigQuery to Databricks is an organizational transition, not just a technical one. This retrospective covers the planning decisions, governance redesign, and cost realities that determine whether a migration delivers value or just progresses — drawn from real implementation experience.

A complete SAP HANA + BO/SAC to Databricks migration guide — business case, TCO model, 5-phase technical playbook, and 30-day action plan. Built from a real Fortune 500 engagement.

Databricks Data + AI Summit 2026 made one thing clear: this is no longer just a data platform. From sub-millisecond reads with Lakehouse//RT to CustomerLake, App Builder, and ZeroOps, the announcements tell a single architectural story. Here's what stood out, and why the most important AI news wasn't about AI at all.

Databricks now supports 12 official Docker base images, and you can build your own on top of them. Custom containers let data engineering teams control the full runtime (including native binaries, enterprise certificates, internal packages, and offline environments) not just Python dependencies. This post walks through how to build, test, and deploy a custom Databricks container, with nine production use cases from compliance runtimes to DuckDB integration.

Unity Catalog catalog commits move Delta table commit coordination from filesystem-level operations into Unity Catalog, making UC the authoritative source of table state. This enables strict concurrency control, attribute-based access for external engines, and multi-statement multi-table transactions — while leaving data in open Delta format on cloud storage.

Declarative Automation Bundles mutators let you define job parameters once in a central config file and inject them into every job automatically at deploy time — no more copy-pasting catalog names, schema paths, and environment variables across your bundle. This post walks through a working implementation with full code examples.

Earning a Databricks BrickBuilder Specialization in Financial Services requires more than technical credentials — it demands domain knowledge built across banks, insurers, and investment firms operating under strict audit and regulatory requirements. SunnyData's specialization reflects years of production-ready implementations where data errors have real consequences. For financial services leaders evaluating data partners, it's a signal worth understanding beneath the surface.

Most Databricks teams using Bundles are only scratching the surface. The real value isn't in the deployment syntax — it's in the engineering discipline Bundles makes enforceable: explicit dependency management, reproducible local environments, automated quality gates, and CI/CD as the only path to production. This post breaks down what a production-ready Databricks project structure actually looks like, and the software engineering practices that make it ship with confidence.

Informatica's new LTS pricing means staying put is no longer free — it's a recurring tax on a platform with a shrinking capability roadmap. This guide covers the full migration path from PowerCenter to the Databricks Lakeflow stack: the CFO-ready business case, a construct-by-construct translation guide, and the five-phase M5 methodology SunnyData runs on every engagement. If your team is weighing IICS versus a full re-platform, the architecture decision and cost model are here.