Fsync or Swim Part 1: Error Handling

Written by Bob Lee.

Welcome to part one of our series on file-based persistence. We’ll talk a lot about Android, but these lessons translate easily to any POSIX-compliant platform.

Writing files correctly is notoriously difficult, on par with programming concurrent data structures. In both cases, one must consider data visibility, [re]ordering of operations, and optimizations in the underlying platform. Just like virtual machine and CPU optimizations demand correct concurrent logic, filesystem and storage hardware optimizations assume correct file I/O logic.

Most programmers should stick to high level libraries like java.util.concurrent and SQLite rather than implement their own data structures, but if layering a FIFO queue on top of a B-tree offends your sensibilities, this series is for you.

Even experts have trouble with file I/O. For example, the Android Developers Blog tells just half of what you need to know about using fsync with rename — they achieve atomicity but not durability, and Android’s libraries make common mistakes like ignoring the result of fsync. Neither the standard Java libraries nor Android’s Java libraries directly enable important functionality like durably renaming files. This series will cover all of these issues.

Now that Android has adopted ext4 as its default filesystem, we have to pay even closer attention, especially in apps like Square where mistakes literally cost money. Modern filesystems mostly achieve their high performance by playing fast and loose, by not actually writing data to storage. This works great for apps that write temporary files and then delete them shortly thereafter but not so great for data the user actually cares to keep. Nowadays, if the user removes their battery or runs out of storage while running faulty code, data loss is a given.

When your app writes to a file, data goes into library, kernel, and hardware buffers, not straight to long term storage. If your program doesn’t actually care whether the data makes it to a platter or flash memory or nowhere beyond RAM, it can continue execution without waiting around for the hardware. The OS will eventually try to flush the data to long term storage, sometimes minutes after you write it.

If you care to keep your data, from Java, you can explicitly flush it to storage using FileDescriptor.sync() or FileChannel.force(), or you can configure [RandomAccessFile](http://developer.android.com/reference/java/io/RandomAccessFile.html#RandomAccessFile(java.io.File, java.lang.String)) to flush every write. These methods all delegate to fsync or something like it under the covers. A successful return from any of these methods indicates that your data is safe and sound.

Historically, filesystems pre-allocated and zeroed out hardware pages. For a variety of reasons, newer filesystems like ext4 and YAFFS default to just-in-time allocation. As a result, an application can request far more pages than the underlying storage can accommodate*, but it won’t receive an out-of-space error until it actually tries to write more data than the hardware can fit. Applications that ignore fsync errors (or exceptions from FileDescriptor.sync() et al.) are ticking time bombs waiting to destroy users’ data.

“If you tell the truth, you don’t have to remember anything.”* — Mark Twain*

Consider an email app. The user types an email and taps “Send.” The app writes the message to flash, starts uploading the message to the network in the background, and then immediately returns control back to the user.

If the user runs out of storage space, the page allocation will fail, and fsync will return an error. If the email app ignores the error and tells the user the message enqueued successfully, the user will think the email went out even though the app lost it completely! Instead, the app should propagate the error to the user so they can free up some space and try again.

*Stay tuned for part two wherein we’ll discuss atomicity. Please submit corrections along with your résumé.*

*I’ve successfully created multi-terabyte files on my Android device. Bob Lee - Profile *Reinventing something as fundamental as paths was hard. The solution we came up with may seem obvious, but there were…*medium.com