Databricks driver sizing impact on cost and performance
Databricks driver sizing impact on cost and performance
As many previous blog posts have reported, tuning and optimizing the cluster configurations of Apache Spark is a notoriously difficult problem. Especially when a data engineer needs to lower costs or accelerate runtimes on platforms such as EMR or Databricks on AWS, tuning these parameters becomes a high priority. Here at Sync, we will experimentally
Jeffrey Chou
07 Feb 2023
Blog, Case Study
Is Databricks autoscaling cost efficient?
Is Databricks autoscaling cost efficient?
Here at Sync we are always trying to learn and optimize complex cloud infrastructure, with the goal to help more knowledge to the community. In our previous blog post we outlined a few high level strategies companies employ to squeeze out more efficiency in their cloud data platforms. One very popular response from mid-sized to
Jeffrey Chou
20 Jan 2023
Blog, Case Study
The top 6 lessons learned why companies struggle with cloud data efficiency
The top 6 lessons learned why companies struggle with cloud data efficiency
Here at Sync, we’ve spoken with companies of all sizes, from some of the largest companies in the world to 50 person startups who desperately need to improve their cloud costs and efficiencies for their data pipelines. Especially in today’s uncertain economy, companies worldwide are implementing best practices and utilizing SaaS tools in an effort
Jeffrey Chou
14 Dec 2022
![]()
Blog
We’re hiring, let’s build.
We’re hiring, let’s build.
What are we building? At Sync we’re building something that is really hard. We’re trying to disrupt a $100B industry where some of the world’s biggest companies live. On top of that, we’re attacking a layer in the tech stack that is mired in complexity, history, and evolution. So why do we think we’re going
Jeffrey Chou
07 Oct 2022
![]()
Blog
Globally Optimized Data Pipelines On The Cloud — Airflow + Apache Spark
Globally Optimized Data Pipelines On The Cloud — Airflow + Apache Spark
Sync Computing presents a new kind of scheduler capable of automatically optimizing cloud resources for data pipelines to achieve runtime, cost, and reliability goals Here at Sync, we recently launched our Apache Spark Autoutuner product, which helps people optimize their EMR and Databricks clusters on AWS. Turns out, there’s more on the roadmap for us
Jeffrey Chou
31 May 2022
![]()
Blog
Rising above the clouds
Rising above the clouds
The case for Sync, and why companies need our solution.
Suraj Bramhavar
03 Jan 2022
![]()
Blog
