Disney Sr. Data Engineer User Case Study
Disney job speedup from 90 min. to 24 min.

Sr. Data Engineer at Disney Streaming
In the self-written blog post below, a Sr. Data Engineer chronicles his experience with the Spark Autotuner for EMR. In the blog post we helped accelerate a job from 90 to 24 minutes, which was amazing to see!


The first job I put into the autotuner went from processing in around 90 minutes to 25 minutes after I changed the configurations, only using a slightly larger cluster. However, that time save makes up for using more nodes, so it definitely worked to our advantage.
Matthew Weingarten

Extrapolated over a full year, our anticipated savings to his company was over $100K on AWS! Obviously this doesn’t include the extra time savings of removing the current manual guesswork of provisioning clusters.

User’s blogpost here
More from Sync:
How poor provisioning of cloud resources can lead to 10X slower Apache Spark jobs

How poor provisioning of cloud resources can lead to 10X slower Apache Spark jobs
How does the worker size impact costs for Apache Spark on EMR AWS?

How does the worker size impact costs for Apache Spark on EMR AWS?
Databricks driver sizing impact on cost and performance

Databricks driver sizing impact on cost and performance