Optimizing Databricks AI/BI Implementation for Enterprise-Scale Analytics
SunnyData partnered with Farmatodo, a leading Latin American pharmacy chain, to optimize their Databricks AI/BI implementation after migrating from Looker. By improving their semantic layer, optimizing underlying tables, and implementing dashboard best practices, SunnyData achieved 60-70% reduction in query times while enabling secure, scalable self-service analytics for 100+ users across the organization.
Key Metrics
INDUSTRY: Retail/Pharmacy
SOLUTION: Business Intelligence Optimization, Semantic Layer Development, Query Performance Tuning, Self-Service Analytics
PLATFORM USE CASE: Databricks AI/BI Dashboards, Metric Views, Delta Lake
Farmatodo is a leading self-service pharmacy chain with presence across Latin America (specifically Venezuela, Colombia, and Argentina). Since its beginnings more than a century ago, they have expanded to include +300 stores, +5k employees, and an annual revenue of $75.7 Million. While Farmatodo had already migrated its architecture to Databricks and adopted AI/BI dashboards, they were struggling to take full advantage of the platform's capabilities and experiencing significant performance issues that were hindering their analytics adoption.
Client Challenges
After migrating from Looker to Databricks AI/BI, Farmatodo found themselves in a frustrating situation. Their dashboards were loading painfully slowly, creating friction with users and limiting adoption across the organization.
The team had recognized the value of Databricks Metric Views for organizing their business metrics and made the decision to implement them, but in the transition, they had essentially replicated their Looker queries without adapting them for the new platform. This meant they were carrying over legacy practices that weren't optimized for Databricks.
The problems ran deeper than just query translation. The underlying tables feeding these Metric Views hadn't been properly optimized and weren't leveraging any of Databricks' native features.
Furthermore, implementing row-level security (a customer requirement) to protect sensitive data, meant sacrificing speed. This made the case of improving overall performance even more essential. Without addressing the underlying optimization issues first, adding security controls would have pushed an already slow performance.
Looking ahead, Farmatodo had ambitious plans to empower approximately 100 business users to create their own dashboards for over 400 viewers across the organization. But without proper guidelines and a performant foundation, this self-service BI vision seemed increasingly out of reach.
Solution Implementation
SunnyData’s team approached the engagement with a clear philosophy: respect the work Farmatodo had already done while introducing the improvements needed to make Databricks truly work for them. The challenge was finding that sweet spot: tightening the bolts and tuning performance without breaking what was already functioning in production. This required deep expertise in Databricks' inner workings and a careful understanding of how far the platform could be pushed to optimize a live system.
Phase 1: Discovery
The team spent time understanding how Farmatodo's business users actually navigated and analyzed their information, particularly in critical areas like Sales. We went beyond technical architecture and understood the business context and workflows that the data needed to support.
Phase 2: Table Redesign
With this understanding in place, SunnyData redesigned the underlying tables to incorporate Databricks-native features that would improve query optimization. The Metric Views were rebuilt following Databricks best practices while carefully maintaining the business logic that Farmatodo relied on. This created a robust semantic layer that could serve as the foundation not just for current dashboards, but for all future BI processes.
Phase 3: Fine-tuning Performance
Our engineers carefully evaluated how security filters impacted query speed and worked to minimize the performance penalty that typically comes with Databricks' access control mechanisms. By implementing Databricks-native features and restructuring the data architecture, SunnyData achieved improvements in query execution times. This created enough performance headroom to absorb the overhead that security controls naturally introduce.
Phase 4: Dashboards Re-design
For the dashboard layer, SunnyData provided architectural guidance that would enable Farmatodo's self-service BI vision. This included guidance on developing dashboard structures that would increase their performance, and training the team on how to properly leverage Metric Views when building visualizations. The goal was to create a sustainable framework that balanced user empowerment with performance.
Key Benefits Achieved
The results were immediate. The longest-running queries’ execution times dropped by 60-70% across dashboards, improving user experience and enabling the broader adoption that Farmatodo had been seeking.
With the performance issues resolved and best practices in place, Farmatodo's vision for self-service analytics became achievable. The company now has a solid foundation for 100+ business users to create and publish their own dashboards, providing analytics access to over 400 end users across the organization.
Perhaps most importantly, Farmatodo now has the right semantic layer, the critical foundation that feeds all BI processes (like feeding dashboards, AI assistants like Databrick’s Genie, and SQL queries without any data duplication). This positions them not just for today's needs, but for strategic growth.
Throughout the implementation, knowledge transfer was paramount. SunnyData worked closely with Farmatodo's data analytics / BI team, building their technical confidence and establishing an ongoing advisory relationship that would extend beyond the initial engagement.
One of the most notable outcomes was the empowerment of d&n’s internal data team. Thanks to comprehensive knowledge transfer during the implementation, the team is now equipped to implement similar AI-driven solutions across other business units, reducing their dependency on external consultants. This internalization of expertise positions d&n for future growth, as they can now scale these capabilities independently across the business.
