Top 3 trends we’ve learned about the scaling of Apache Spark (EMR and Databricks)
Top 3 trends we’ve learned about the scaling of Apache Spark (EMR and Databricks)
We launched the Gradient for Apache Spark several months ago, and have worked with many companies on analyzing and optimizing their Apache Spark workloads for EMR and Databricks. In this article, we summarize cluster scaling trends we’ve seen with customers, as well as the theory behind it. The truth is, cluster sizing and configuring is
Jeffrey Chou
02 Aug 2022
Case Study
Globally Optimized Data Pipelines On The Cloud — Airflow + Apache Spark
Globally Optimized Data Pipelines On The Cloud — Airflow + Apache Spark
Sync Computing presents a new kind of scheduler capable of automatically optimizing cloud resources for data pipelines to achieve runtime, cost, and reliability goals Here at Sync, we recently launched our Apache Spark Autoutuner product, which helps people optimize their EMR and Databricks clusters on AWS. Turns out, there’s more on the roadmap for us
Jeffrey Chou
31 May 2022
![]()
Blog
Optimize Databricks Clusters Based on Cost and Performance
Optimize Databricks Clusters Based on Cost and Performance
Databricks is increasingly one of the most popular platforms to run Apache Spark, as it provides a relatively friendly interface that allows data scientists to focus on the development of the analytical workloads—and efficiently build extract load transform (ELT) type operations. The multiple options it provides, by virtue of being built on top of Apache
Jeffrey Chou
10 Apr 2022
Case Study
Distributed Cloud Infrastructure Innovator Sync Computing Emerges from Stealth, Brings in $6.1 Million
Distributed Cloud Infrastructure Innovator Sync Computing Emerges from Stealth, Brings in $6.1 Million
First to launch predictive gradient solution for big data jobs in the cloud, wins $1M contract from the Department of Defense BOSTON, Jan. 25, 2022 (GLOBE NEWSWIRE) — Sync Computing, a deep tech, distributed cloud infrastructure company, came out of stealth mode today, revealing its initial products, customer traction, and $6.1 million funding. Moore Strategic
Jeffrey Chou
27 Jan 2022
News
Rising above the clouds
Rising above the clouds
The case for Sync, and why companies need our solution.
Suraj Bramhavar
03 Jan 2022
![]()
Blog
How Duolingo reduced their EMR job cost by 55%
How Duolingo reduced their EMR job cost by 55%
Launch to the cloud based on cost and time: This article explains Sync Computing’s Spark Cluster Gradient Solution and how it was used to reduce Duolingo’s AWS EMR Spark costs by up to 55%. This solution eliminates the inefficient manual tuning and guesswork currently used when configuring Spark clusters and settings to provide the best
Jeffrey Chou
06 Dec 2021
Case Study
