How to evaluate the performance of your Databricks Jobs clusters with Gradient
How to evaluate the performance of your Databricks Jobs clusters with Gradient
Many data engineers and platform managers at companies want to understand “How are my pipelines doing from a cost and performance perspective?” It turns out this very benign and simple question is riddled in complexity. Fundamentally there are so many things that can change from run to run, that even determining if it’s the same
Jeffrey Chou
03 Sep 2024
Blog, Case Study
Meet the team at Databricks Data + AI World Tour in London and New York City!
Meet the team at Databricks Data + AI World Tour in London and New York City!
Sync computing is a proud sponsor of Databricks Data + AI World Tour in London and New Your
Noa Shavit
15 Aug 2024
![]()
Blog
Best Practices to Manage Databricks Clusters at Scale to Lower Costs
Best Practices to Manage Databricks Clusters at Scale to Lower Costs
Managing Databricks clusters efficiently is a critical challenge for many organizations, especially as they scale on the platform. As the amount of data organizations process grows rapidly, so do compute costs. Most organizations tackle this by having their data engineers manually optimize clusters, which quickly becomes a bottleneck as the organization scales from dozens to
Noa Shavit
08 Aug 2024
![]()
Blog, Videos
Databricks 101: An Introductory Guide on Navigating and Optimizing this Data Powerhouse
Databricks 101: An Introductory Guide on Navigating and Optimizing this Data Powerhouse
In an era where data reigns supreme, the tools and platforms that businesses utilize to harness and analyze their data can make or break their competitive edge. Among these, Databricks stands out as a powerhouse, yet navigating its complexities often feels like deciphering cryptic code. With businesses generating an average of 2.5 quintillion bytes of
Noa Shavit
16 Jul 2024
![]()
Blog
Top 9 Lessons Learned about Databricks Jobs Serverless
Top 9 Lessons Learned about Databricks Jobs Serverless
To much fanfare, Databricks announced their wide release of serverless compute across all of their platforms at the Data + AI 2024 conference. It’s quite clear that Databricks’ vision is to own the compute layer to make life easier for end users, so they don’t have to worry about annoying cluster or versions details. We
Jeffrey Chou
02 Jul 2024
![]()
Blog
Gradient Product Update— Discover, Monitor, and Automate Databricks Clusters
Gradient Product Update— Discover, Monitor, and Automate Databricks Clusters
With the Databricks Data+AI Summit 2024 just around the corner, we of course had to have a major product launch to go with it! We’re super excited to announce an entirely new user flow and features to the product, making it faster to get started and providing a more comprehensive management solution. At a high
Jeffrey Chou
06 Jun 2024
![]()
Blog, Releases
