The Easy and Comprehensive Guide To Understanding Databricks Pricing: How It Works and How To Reduce Your Cost
The Easy and Comprehensive Guide To Understanding Databricks Pricing: How It Works and How To Reduce Your Cost
Databricks is a popular unified analytics platform and a go-to solution for many organizations looking to harness the power of big data. Its collaborative workspaces have become the industry standard for data engineering and data science teams and an ideal environment for building, training and deploying machine learning and AI models at scale. However—as with
Jeffrey Chou
28 Sep 2023
Blog
New Gradient quick-start notebooks- Optimize your Databricks jobs in minutes
New Gradient quick-start notebooks- Optimize your Databricks jobs in minutes
Since we launched Gradient to help control and optimize Databricks Jobs, one piece of feedback from users was crystal clear to us: “It’s hard to set up” And we totally agreed. By nature, Gradient is a pretty deep infrastructure product that needs not only your Spark eventlogs but also cluster information from your cloud provider.
Pete Tamisin
22 Sep 2023
Blog
Why Your Data Pipelines Need Closed-Loop Feedback Control
Why Your Data Pipelines Need Closed-Loop Feedback Control
Realities of company and cloud complexities require new levels of control and autonomy to meet business goals at scale
Jeffrey Chou
10 Sep 2023
Blog
Are Databricks clusters with Photon and Graviton instances worth it?
Are Databricks clusters with Photon and Graviton instances worth it?
Configuring Databricks clusters can seem more like art than science. We’ve reported in the past about ways to optimize worker and driver nodes, and how the proper selection of instances impacts a job’s cost and performance. We’ve also discussed how autoscaling performs, and how it’s not always the most efficient choice for static jobs. In
Jeffrey Chou
17 Aug 2023
Blog
How to Use the Gradient CLI Tool to Optimize Databricks / EMR Programmatically
How to Use the Gradient CLI Tool to Optimize Databricks / EMR Programmatically
Introduction: The Gradient Command Line Interface (CLI) is a powerful yet easy utility to automate the optimization of your Spark jobs from your terminal, command prompt, or automation scripts. Whether you are a Data Engineer, SysDevOps administrator, or just an Apache Spark enthusiast, knowing how to use the Gradient CLI can be incredibly beneficial as
Pete Tamisin
11 Jul 2023
Blog, Case Study
