Getting Square's iOS build ready for Apple Silicon with Bazel

Introduction to iOS at Square

At Square we use Bazel to build our iOS applications. Bazel is a build tool with an emphasis on performance and correctness. Most of our iOS app code lives in a single git repository. The repository includes over one thousand modules, contains Objective-C and Swift, and includes internal and external binary dependencies. Square adopted Bazel to improve performance, make the IDE more responsive, and gain control of builds. Bazel reduced build times by up to 4x in some cases. Bazel's build language, robust open source core, and extensibility model gives us full control over the entire system including the build graph and execution. This enabled key improvements such as rolling out fine-grained build and test caching backed by a CDN, optimizing compiler invocations, better architecting our CI, and easily running on the Apple Silicon simulator. The core build rules are community based and open sourced at rules_ios. Many thanks to Oscar Bonilla and Sam Giddins for open sourcing rules_ios.

Square up and running on the M1 Simulator.

Apple Silicon for iOS developers

In June 2020, Apple announced the Mac was transitioning to an Arm based CPU known as Apple Silicon. While we’re excited about the performance benefits of Apple Silicon, this post is focused on the iOS Simulator. The simulator is the primary tool which iOS developers use to iterate, build, and run code. Like other macOS applications, it executes locally on the developer’s OS and CPU. With the Apple silicon, you can also run Arm iOS apps directly on the simulator! The Arm instruction set architecture (ISA) plays a key role in enabling this:

“The Arm ISA family allows developers to write software and firmware that conforms to the Arm specifications, secure in the knowledge that any Arm-based processor will execute it in the same way."

Per arm developer https://developer.arm.com/architectures/instruction-sets

For the last decade, the Mac has used Intel CPUs and the simulator executed x86_64 ISA builds of iOS applications. The Apple Silicon transition meant Intel built simulator apps would run under a transition layer, Rosetta 2.

Shortest path to the M1 Simulator

The three possible paths to using M1 for development include running device builds, using x86_64 builds under Rosetta 2, and building for the M1 simulator. Because a large segment of the app source works differently when built for the device, just using device builds was ruled out. The Rosetta 2 transition layer was ruled out because of runtime issues and the 2 year transition period to Apple Silicon. Building and running on the M1 simulator was the approach we landed on. Going M1 meant updating iOS dependencies to support Arm simulator builds.

To deliver device and simulator dependencies in a single file, iOS binary dependencies are often packaged as a fat binary. A fat binary is a build artifact which is partitioned by architecture. Linkers can select the right slice from a fat binary for a given architecture. Traditionally, systems used an arm64 slice for device and an x86_64 slice for simulator. This no longer works when both the simulator and device are also arm64. Apple introduced the xcframework structure which standardizes dependency packaging and helps support Apple Silicon. A typical xcframework includes an Info.plist which points to the correct binary.

Triaging the company wide update with Bazel

To fully support the M1 simulator, a mass upgrade of binary dependencies was in-order. This was a necessary step because some of the dependencies have different behaviors under the simulator. Bazel query is one of Bazel’s power tools that helped triage. Bazel query is a tool that includes a query language and command line interface to print information about the build. To quickly find and triage updates, we leveraged it to find all of the binary dependencies and then triaged JIRA tickets to owners. At Square, repos contain an OWNERS file; this made it easy to map dependencies to the right teams.

bazel query ‘kind(import, ...)’ # prints all of the imported binaries in the tree

While the mass update was under-way, we wanted to build and run the app to ensure it worked and find further issues. We didn’t want to wait to use the M1 simulator until all dependencies were updated.

An interim solution to use the M1 simulator

Because the iOS platform also uses the Arm ISA, device code can run natively on the Mac. We can leverage the fact that many dependencies would operate similar enough for most developers to use the simulator. A large portion of iOS developer code is functionally the same when built for the simulator. Examples of exceptions include platform specific compiler flags such as TARGET_IS_SIMULATOR and platform specific dependencies. We just needed to fix build-time and run-time errors.

Statically linked dependencies

In an iOS build, static frameworks and static libraries are linked into an app bundle at build time. When the linker encounters a static dependency that was built for device, it raises an error:

 error: Building for simulator, but linking in file '' built for ios 

In Apple’s linker, ld64, this is a platform mismatch error and a concern of the linker. In order to get the application building, this linker error needs to be suppressed. There are a couple of options: building a linker that ignores it, arm64-to-sim can update headers in position independent object files, and Apple’s linker ld provides an undocumented flag to map the error into a warning. We originally prototyped the first version of this solution using arm64-to-sim approach and stuck with it to avoid an undocumented API, avoid turning it off globally, and avoid adding warnings. Additionally, we needed to rewrite macho headers for dylibs which added the requirement to patch binaries. The undocumeted ld flag is a quick fix if for a build that only has static dependencies, don’t mind disabling it globally.

Dynamically linked dependencies

In an iOS application, dynamic libraries and dynamic frameworks are packaged into the .app directory so the system can load them at runtime. Running an application in the simulator with dynamic device dependencies raises an error:

 Application launch for '' did not return a valid pid nor a launch error.

To get around this, we can simply update the macho headers to reflect a simulator built binary. Apple provides the vtool program that can achieve this. Credit to Hacking native Arm64 binaries to run on the iOS Simulator - the dynamic framework edition post which documents the problem space and leveraging vtool for a fix.

Automatic M1 dependency conversion with Bazel

With the power of Bazel we are able to easily get up and running by converting device dependencies to work with M1. Bazel provides abstractions like rules and aspects that allow us to get the job done correctly and integrate in with our existing build rules.

The gist of the idea is to find all of the dependencies in the build graph that are pre-compiled and patch them to work with the M1 simulator. Bazel makes this easy with primitives such as rules, aspects, and actions. In Bazel a rule is a function to define build logic that operates on inputs and produces outputs. An aspect allows augmenting the build graph with additional information or actions. Under rules_ios, an iOS application is typically implemented as an ios_application rule that depends on an apple_framework rule which depends on source code and precompiled dylibs.

Simplified representation of an iOS application’s dependency graph

For Apple Silicon support we introduced a “middleman” abstraction to provide dependencies to an ios_application's executable linker invocation. Under this scheme, an ios_application depends on an import_middleman which applies the find_imports aspect to dependencies. The find_imports aspect aggregates dependencies into a set, and for each of the items an action is created to update. For dynamic libraries the action runs vtool and for static archives it runs arm64-to-sim.

Simplified representation of an iOS application’s dependency graph plus the middleman abstraction

Conclusion

There will be a long tail of internal and external binary dependencies being updated to the new xcframework package. Bazel made it possible to both triage the updates and provide an interim solution. Because we have many internal and external dependencies, having an interim solution was an essential step across the company in our path.

If you’re interested in improving iOS builds with Bazel, come and join our team. Search careers at Block.