Saving 5,400 hours a year with Gradle's Configuration Cache
Recovering an estimated $1.1 million in lost productivity annually
Have you ever had to wait for some ridiculously lengthy process to validate the completion of your work? This lengthy wait time is something our Android developers can relate to! They could watch 60 back-to-back football games in the amount of time they collectively wait for the compilation of their code on their laptops, also known as the local build process. For perspective, the Mobile Developer Experience (MDX) Android team supports about 200 developers who contribute code to a repository with nearly 4 million lines of code spread across more than 4,000 modules. This equates to 182 hours per week in time spent waiting on local builds to complete. A snail could make its way up and down an entire football field in this much time!
MDX Android has historically had many priorities, but decreasing local build times became a primary focus when it became clear that it would improve developer productivity and unlock new efficiencies. But first, one may wonder why these build tools take so long to work in the first place. The reason is that Gradle, the build tool for building our Android apps, has 3 major phases:
-
Initialization Phase: Builds a graphical representation of tasks (chunks of work) for each project.
-
Configuration Phase: Downloads dependencies, applies plugins, and creates model representations of each modules’ task set. This produces a graph of tasks for phase 3 to consume.
-
Execution Phase: Runs the subset of tasks based on the input from the graph built in phase 2.
The Configuration Phase comes with heavy computation time spent on recomputing the graph of tasks to run each time the code is recompiled. With our repository containing more than 4,000 modules, the time associated with building this graph is high. Gradle realized this overhead of their build process, and in 2019 began iterating on the Configuration Cache functionality, which allowed the configuration phase to reuse previous computations of their task graphs. At Square we saw that the integration of this feature decreased our local build times from 182 hours to 25 hours per week!
The integration of this feature was by no means an easy feat. On the surface, the Configuration Cache may sound like an on and off button for our build tool infrastructure, but in reality, its implementation presented numerous challenges. We had to make convincing arguments to multiple third parties to upgrade their software to be compatible with our repository. To that end, Square also contributed to addressing concerns with our open source software, such as Anvil and Wire. For example, newer versions of Wire were not compatible with the configuration cache, which led to a fork of the code that the Foundations team helped resolve!
MDX Android has written a suite of tools that automate common tasks to ease the lives of our feature developers. We have incorporated the ability to enable the configuration cache to make the transition easy for all of our developers. Based on the results of this opt-in approach, we are now planning to enable the configuration cache by default while permitting engineers to opt-out on a case-by-case basis.
Results
-
With the configuration cache on, local builds averaged about 74 seconds, whereas with the configuration cache off those builds took about 136 seconds. Enabling configuration cache therefore improves build performance by 183.67%.
-
Builds running the same task graph (or same input) saw a 35 second decrease in build time when running with configuration cache enabled.
-
Builds that failed were able to fail 300% faster with configuration cache enabled
Key Takeaways
-
Net savings of > 5,400 hours per year if we enable configuration cache for all builds
-
By having faster failures we decrease the time it takes to find failing tests, and iterate on those issues.
-
We did not see an increase in failures for builds using the configuration cache. This gives strong indication that it can scale well.
We calculate (based on industry averages found in Glassdoor) that annually, we can save over 3 years worth of engineering time currently being allotted to local builds, accounting for over an estimated $1.1 million in lost productivity per year!
Time spent on builds NOT using configuration cache in the second week of August
Time spent on builds using configuration cache in the second week of August
It is clear that enabling the configuration cache on local builds ensures an improved developer experience. With the expected time saving at around 5,400 hours annually and builds reaching an eventual outcome at 3 times faster, the next step for our team is to enable Configuration Cache by default for our entire repository.
Authored By
