Databricks Summit ’23 Talk: “How we built Gradient to optimize Databricks clusters at scale”
Databricks Summit ’23 Talk: “How we built Gradient to optimize Databricks clusters at scale”
Jeffrey Chou30 Jun 2023
Blog
Jeffrey Chou30 Jun 2023
Sync Blog & News
Stay in sync
with Sync.
Sync Blog
Jeffrey Chou30 Jun 2023
Integrating Gradient into Apache Airflow
Integrating Gradient into Apache Airflow
Summary In this blog post, we’ll explore how you can integrate Sync’s Gradient with Airflow. We’ll walk through the steps to create a DAG that will submit a run to Databricks, and then make a call through Sync’s library to generate a recommendation for an optimized cluster for that task. This DAG example can be
Brandon Kaplan27 Jun 2023
Blog
Developing Gradient Part II
Developing Gradient Part II
Introduction: Using Gradient in a Workflow Gradient, the latest product release from Sync Computing, helps customers manage the infrastructure behind their recurring Apache Spark applications. Gradient gives infrastructure recommendations for each job to lower the cost of their Production jobs while hitting their target SLA’s. We’ve been hard at work on this project for a
Sean Gorsky19 Jun 2023
Blog
Developing Gradient Part I
Developing Gradient Part I
Introduction Sync recently introduced Gradient, a tool that helps data engineers manage and optimize their compute infrastructure. The primary facet of Gradient is a Project which groups a sequence of runs of a Databricks job. After each run, the Spark eventlog and cluster information is sent to Sync. That accumulated project data is then fed
Sean GorskyBlog
Introducing: Gradient for Databricks
Introducing: Gradient for Databricks
Wow the day is finally here! It’s been a long journey, but we’re so excited to announce our newest product: Gradient for Databricks. Checkout our promo video here! The quick pitch Gradient is a new tool to help data engineers know when and how to optimize and lower their Databricks costs – without sacrificing performance.
Jeffrey ChouBlog
How Insider Reduced their EMR Cost by 25%
How Insider Reduced their EMR Cost by 25%
Insider’s engineering blog discusses how they integrated the Sync Gradient API into their Airflow pipelines to continuously monitor and reduce costs.
Jeffrey Chou09 Apr 2023
Case Study
