Resources and insights
Our Blog
Explore insights and practical tips on mastering Databricks Data Intelligence Platform and the full spectrum of today's modern data ecosystem.
From Model to Production: Building a Governed ML Lifecycle on Databricks
Most ML teams can train a model, but far fewer can operate one. This post shows how to build a governed ML lifecycle on Databricks, with automated validation, champion-challenger promotion, one-swap rollback and continuous monitoring, using MLflow and Unity Catalog.
Environment Variables in Databricks Serverless
Copying the same job for dev, staging, and prod invites drift and leaked secrets. Here’s how serverless environment variables let one job definition carry different settings per task, with a working three-task demo.
Automatic Change Data Feed (CDF) is here to save space, improve write time, and feed Change Data Capture (CDC)
Classic change data feed saves an extra copy of every change, which adds storage and slows updates. Hubert Dudek benchmarks the automatic alternative on a 5-million-row table and walks through how it works.
Beyond Bronze, Silver, and Gold: Designing Medallion Layers Around Real Data Needs
Bronze, Silver, and Gold are a great starting point, but real projects rename layers, split them, and stack them. Here’s how to decide what each layer is responsible for, and when a separate one earns its place.
Databricks DABs CI/CD: 4 Ways to Automate Your Deployments
Every CI/CD pipeline eventually needs something to run after deployment: initial setup, config syncs, validation checks. job_runs in Databricks Asset Bundles handles all of it natively, and you can use it to block a production update when validation fails.
Automated Tags Clean The Mess
Learn how to use Databricks Unity Catalog's governed tags to automatically flag undocumented or inactive tables, notify owners, and build a self-governing data catalog.
Databricks as a Model Provider: The Safe Route into Enterprise Adoption of Open-Weight Models
Nicanor Medina (Databricks MVP) explains how Databricks is emerging as a top-tier inference provider for open-weight models like Kimi K3 and GLM-5.3. Fast, affordable, and enterprise-secure.
Genie ZeroOps: A Hands-On Preview
A hands-on preview of Genie ZeroOps, Databricks' autonomous agent for monitoring and fixing data pipeline issues — tested against a data overflow bug, a division-by-zero error, and a live migration.
DABs direct mode: available, faster, and better.
Databricks Asset Bundles direct mode is now GA, and it's already the default for bundles created from the workspace, with full CLI default rollout on August 26, 2026. By removing the Terraform dependency, direct mode cuts deployment time by up to 40%, replaces opaque state with human-readable JSON, and unlocks resources like instance pools and Genie Agents that Terraform-gated releases used to delay.
Databricks Zerobus vs. Kafka: Do you still need Kafka in front of your Lakehouse?
Does 'streaming means Kafka' still hold for single-sink ingestion into Delta? We benchmarked it with 64.8 million real events and found a cost gap that never closes, 3.6× to 20.9×
I benchmarked the new Lakehouse RT: It’s fast, cheap, and open-format
Most teams assume faster real-time queries mean higher compute costs. We benchmarked Databricks' new RT Lakehouse engine across datasets from 1 million to 50 billion records to find out, and the results reset what "fast" costs on a lakehouse.
The Translator’s Gone: Inside Databricks' Direct Engine
Databricks just replaced the engine behind DABs deployment. Here's what Direct Engine changes, why it matters, and how to migrate your existing bundles before Terraform support goes away.
Full-text search index on Databricks. How we cut search time by 400x.
We benchmarked Databricks' new full-text search index across tables from 100M to 10B rows. The result: a 10B-row word search dropped from 71 minutes to under 11 seconds: a 99.75% speedup! Here's how the feature works, how to set it up, and when it's not worth using.
BigQuery to Databricks: What the Migration Actually Looks Like
Migrating from BigQuery to Databricks is an organizational transition, not just a technical one. This retrospective covers the planning decisions, governance redesign, and cost realities that determine whether a migration delivers value or just progresses — drawn from real implementation experience.
SAP HANA to Databricks Migration: The Complete Guide
A complete SAP HANA + BO/SAC to Databricks migration guide — business case, TCO model, 5-phase technical playbook, and 30-day action plan. Built from a real Fortune 500 engagement.