Transparency with JPEGs

Written by Christian Williams and Kiran Ryali.

We have some of the best — and most demanding — designers in the world at Square.

While pushing the bounds of design, we tend to push the bounds of the platform. For the new version of Wallet, our designers asked for a landing page with high resolution transparent images. These images would animate over each other creating a realistic parallax effect.

Our first thought was to use a PNG so we could have a high-fidelity image with an alpha channel for transparency. However, because PNG is lossless, the image was a staggering 753 KB. For one image. That appears in the app once and never again. This single image would be a third of the app download size. Ouch.

There are a few tools out there that we thought could help:

• ImageOptim is our favorite tool for squeezing every last byte out of an image while still preserving image quality. Unfortunately, it didn’t help much here: it shrank the file a bit, but not enough.

• TinyPNG is an online tool that sounded promising. The tool gave us a savings of 81%! However, image quality was reduced substantially: the image was converted from 24 bits to 8 bits, resulting in noticeable color banding.

So PNGs, while offering transparency, just aren’t an efficient format for photographic images. By comparison, the same image saved as a JPEG is only 64K! Really what we want is a JPEG with transparency.

Computer says no.

The JPEG format doesn’t support transparency. But we can create our own transparency using a second image as an alpha channel. Since this image is simple and monochromatic, it compresses extremely well: ours came out to just 11K.

When we render the image, we load two images into memory: the full-color JPEG and the monochrome alpha channel. For each pixel, we take color information from the first image, and transparency information from the second. A white pixel in our alpha channel image indicates fully opaque, while a black pixel indicates completely transparent. Shades in between indicate the corresponding level of transparency. Combining the color information and the transparency, we can create a composite bitmap. We provide the graphics library with a 32-bit bitmap (RGB+A) and it handles the rest.

This technique is platform independent, but we will show an implementation for the Android platform.

Implementation

To create a transparent image, we need to show Android how to composite a bitmap from our two assets.

Here was our first take at the compositing code:

**for** **(int** y **=** 0**;** y **<** height**;** y**++)** **{**
  **for** **(int** x **=** 0**;** x **<** width**;** x**++)** **{**
    **int** rgbPixel **=** rgbBitmap**.**getPixel**(**x**,** y**);**
    **int** alphaPixel **=** alphaBitmap**.**getPixel**(**x**,** y**);**
    *// Replace the alpha channel with the value from the bitmap.*
    **int** compositePixel **=** **(**rgbPixel **&** 0x00FFFFFF**)**
                         **|** **((**alphaPixel **<<** 8**)** **&** 0xFF000000**);**
    destBitmap**.**setPixel**(**x**,** y**,** compositePixel**);**
  **}**
**}**

It works! And it looks great, but we found that it was pretty slow. There are three method calls for every single pixel. In our case, that was more than 2 million method calls. Method dispatch is pretty expensive in any language, so even though each call was very fast, it added up to over 300ms to simply composite the image. Too slow!

Luckily, most graphics libraries provide a way to work with groups of pixels at once. Android provides a [getPixels](https://developer.android.com/reference/android/graphics/Bitmap.html#getPixels(int[], int, int, int, int, int, int)) method which returns an array of pixels. We can then operate on all the pixels without invoking any methods. We decided to ask Android for a row’s worth of pixels at a time, which reduced the number of method calls to about 3,000, or less than 3ms.

**public** **static** BitmapDrawable **compositeDrawableWithMask(**Resources resources**,**
    BitmapDrawable rgbDrawable**,** BitmapDrawable alphaDrawable**)** **{**
  Bitmap rgbBitmap **=** rgbDrawable**.**getBitmap**();**
  Bitmap alphaBitmap **=** alphaDrawable**.**getBitmap**();**
  **int** width **=** rgbBitmap**.**getWidth**();**
  **int** height **=** rgbBitmap**.**getHeight**();**
  **if** **(**width **!=** alphaBitmap**.**getWidth**()** **||** height **!=** alphaBitmap**.**getHeight**())** **{**
    **throw** **new** **IllegalStateException(**"image size mismatch!"**);**
  **}**

  Bitmap destBitmap = Bitmap.createBitmap(width, height,
      Bitmap.Config.ARGB_8888);

  int[] pixels = new int[width];
  int[] alpha = new int[width];
  for (int y = 0; y < height; y++) {
    rgbBitmap.getPixels(pixels, 0, width, 0, y, width, 1);
    alphaBitmap.getPixels(alpha, 0, width, 0, y, width, 1);

    for (int x = 0; x < width; x++) {
      // Replace the alpha channel with the r value from the bitmap.
      pixels[x] = (pixels[x] & 0x00FFFFFF) | ((alpha[x] << 8) & 0xFF000000);
    }
    destBitmap.setPixels(pixels, 0, width, 0, y, width, 1);
  }

  return new BitmapDrawable(resources, destBitmap);
}

Building the Mask

We used Photoshop to create the alpha channel mask image. Starting with our transparent source image, here’s how we did it:

1. Save the color image as a JPEG.

2. Ctrl/⌘ + Click on the layer icon to make a selection of your transparent image.

3. Switch over to the channels tab and click Save Selection as Channel

4. Enable the new channel and unselect the others.

5. Set the image mode to grayscale and flatten all the layers.

6. Save the grayscale image as a JPEG.

7. Run both images through ImageOptim for extra squeeziness.

Using this compositing method, we’ve shrunk down our image asset size from 753KB to 75KB. That’s a savings of 90%, while still preserving image quality and not sacrificing performance. We win!

Note: Romain Guy, a lead Android framework engineer, pointed out an alternate implementation for dynamic masks which shifts computation to the GPU resulting in saved memory. If your mask is not dynamic, it is even faster and simpler to use Porter-Duff blending modes. Christian Williams *Follow the latest activity of Christian Williams on Medium. 71 people are following Christian Williams to see their…*medium.com Kiran Ryali (@kiranryali) | Twitter The latest Tweets from Kiran Ryali (@kiranryali). Don't trust anything I say with a straight facetwitter.com